Data collection methods are required to collect information on SME performance. These can be quantitative (relating to quantities, measured in numbers) or qualitative (relating to qualities, measured in terms of opinions or degree), or a mix of the two. The KPI Inventory suggests Means of Verification (MoV), which indicate where and how information about the indicator can be obtained. In some instances more than one MoV may be required to cross-validate information or to capture different opinions, particularly for subjective indicators. This is known as triangulation.
To gather each MoV, a range of concrete data collection tools are available.
Producers. These are farm operators (owner-operator, tenant or sharecropper) who may be individual owners, or organisations such as a cooperative. Data is usually collected through surveys, interviews or records. This MoV is only relevant for the agriculture sector.
Workers – Everyone engaged by a company to carry out work, including full and part time employees, contractors and those who receive payment in kind. Data is usually collected through surveys, focus groups, interviews and observation.
Management – People responsible for planning, organising and controlling processes and people, including immediate line managers, middle management, directors and business owners. Data is usually collected through interviews and surveys.
Company records – Written documents such as pay records, personnel records, employment policies/procedures. Data is collected through examining secondary sources, but can be validated through management interviews and worker focus groups.
Data Collection Tools
Data collection tools are the actual instruments used to gather information about the KPIs – such as surveys and interviews. This section provides an overview of the major tools relevant to SME performance measurement.
This quite simply means watching what happens in SME workplaces. Direct observation is undertaken in person by physically visiting the business premises, while indirect observation takes place when using technology such as video recording.
Uses pre-existing sources with the intention of collecting independently verifiable information. These are mostly documents, data files, log sheet or other written pieces such as:
- Pay records
- Personnel Records (e.g. age register, contracts)
- Production schedules
- Sales records and invoices
- Policies and procedures
- Accident and injuries log
- Training records
- Meeting minutes
Interviews are undertaken with individuals with first-hand knowledge of the issue, called key informants. These interviews tend to be loosely structured, relying on a list of issues to be discussed. They resemble a conversation, allowing a free flow of ideas and information. Interviewers frame questions spontaneously, probe for information and take notes – which are elaborated on later. They are useful for collecting in-depth and detailed qualitative data. In SME performance measurement, interviews are particularly useful to explore change processes, and to gather subjective forward-looking ‘lead’ indicators.
A specially selected group is interviewed by a moderator. The group is usually composed of six to twelve individuals. Focus groups explore norms, beliefs, attitudes and practices. They are useful for getting consensus on a topic, or to get a common view of attendants. As they are a ‘group’ discussion, focus groups can lack nuance and risk hiding outlier opinions, as participants may be unwilling to express contrarian views in front of their peers. Focus groups should therefore be complemented with other methods for potentially sensitive issues.
A survey collects data from a large number of people, using a standardized set of questions. There are two common types of survey for SME performance measurement: enterprise and worker.
Enterprise surveys are the most common way to collect data from enterprises. They can be administered to small or large numbers of enterprises, and can last from 15 minutes up to several hours, depending on the amount of information that is required from participants. Most surveys will use a questionnaire to determine which data will be collected.
Enterprise surveys can be administered in many different ways. A self-administered questionnaire refers to a questionnaire that has been designed to be completed by a respondent without intervention of an interviewer collecting the data. Examples of self-administered questionnaires include mail and internet-based surveys.
Self-administered surveys are cheaper as no interviewer is required. Respondents have more time to complete the survey and control the pace at which they do so, making it easier and more convenient for them to respond. Disadvantages include low response rate as survey recipients might not return the survey on time or at all. Data quality might also suffer as recipients might skip or misunderstand questions with nobody to clarify questions or encourage respondent to provide a meaningful answer.
In a person-administered survey, an interviewer reads questions to the respondent, either face to face or over the phone, and records his or her answers. This is the easiest method for the respondent. All they have to do is answer the questions and no additional work is required. The interviewer can adapt the survey for each respondent. Additional clarification about the questions, if needed, can be offered and each question can be answered. Disadvantages include slow speed and higher costs as interviewers need to be recruited and paid to administer the survey.
Certain aspects of working conditions such as wages and working hours can be objectively measured. However, other aspects of working conditions, for example employee satisfaction, are more subjective. Worker surveys can be used to capture different dimensions of working conditions and worker perceptions of them.
A checklist a form that is used to quickly and easily record data. It is particularly effective at registering the occurrence of incidents, events, tasks, or problems. Checklists are often binary – yes / no, exists / does not exist, in place / not in place, and so are often favoured by external auditors rather than as an internal continual improvement tool. Checklists are frequently used during social audits – a variation on an enterprise survey – which is used to check on compliance with performance criteria relating to labour and working conditions. Such audits mostly take place in SMEs part of global supply chains, particularly in consumer-goods industries like the garment and sportswear industry.