R. Gai Murphy1, Norma J. Ford1, Helen Casstles1
1 School of Environment and Life Sciences, University of Salford.
Correspondence: Dr RG Murphy, School of Environment and Life Sciences, University of Salford, M6 6PU, UK. Telephone: 0161 295 5574
In 1996 the Health and Safety Commission issued guidance to Local Authorities which included a requirement that they had a trained and competent inspectorate. Considerable interpretative skills are required in the implementation and enforcement of health and safety legislation and the recent report by the Better Regulation Task Force on Enforcement (Cabinet Office, 1999) emphasises the importance of a trained and competent inspectorate, particularly with regard to consistency of approach. This paper reports on research commissioned by the Local Authority Unit of the Health and Safety Executive to investigate the role emerging technologies can play in assisting Local Authorities to achieve a trained and competent inspectorate, and to assess the acceptability of computer based training as a delivery medium for Local Authority health and safety enforcement officers. Three computer based training packages were developed and piloted within LAs. The packages were delivered through different forms of computer technology, namely floppy disk, CD-ROM and the Internet. Fifty nine enforcement officers tested the packages and were asked detailed questions regarding the ability of the media to deliver effective training in terms of its reliability, usability and flexibility. In addition, line managers from twelve of the LAs which had participated in the pilot were questioned regarding the acceptability and economic viability of delivering health and safety enforcement training in this way. All three packages were enthusiastically received by both trainees and managers. The findings demonstrate that computer based training (CBT) can provide an effective means of delivery of health and safety training to enforcement staff in the workplace and has the facility (assuming the content remains current) to enable several groups to be trained over a period of time. However it is also important to recognise the demands which CBT introduces, both to the trainees and their managers. Shifting to workplace training requires systems to organise and oversee the achievement of learning objectives as trainees progress through the programme(s) offered. Failure to recognise the resources and support required in managing workplace training may result in alienation and a failure to achieve the learning objectives set.
Key Words: Computer based training; CD rom: environmental health, health and safety enforcement officers; Internet; professional development; work based training; web based training.
The training of health and safety inspectors, particularly those employed in LAs, has received considerable scrutiny. The Department of Trade and Industry’s ‘Review of the Implementation and Enforcement of European Law’ (1993) (the DTI Review) suggested that inadequate training of enforcement staff was one of the reasons underlying inappropriate enforcement decisions and the review made recommendations regarding the need to improve training for enforcement staff.
Resource limitations and the inappropriate content of many of the available courses were identified as factors influencing the quality of training provision for LA based health and safety enforcement officers (Prince et al., 1994) Moreover, it was suggested that proper management of training might offer scope for savings on the cost of piecemeal and unintegrated training which could be redirected to more focused training activity. Prince et al. (1994) also drew attention to the difficulty that independent LAs experience in co-ordinating their responses to identified training needs.
The Local Authority Unit (LAU) within HSE commissioned research to examine the problems faced by LAs in addressing these training issues. A fundamental element of the research was to investigate the acceptability to LA based health and safety enforcing officers of training delivered via emerging and novel technologies.
This paper reports on the development of three computer based health and safety training packages, evaluates the pedagogical, ergonomic and economic aspects of delivering training to Local Authority health and safety enforcement officers and explores the opportunities and barriers faced by those wishing to implement computer-based training.
To consider the different types of CBT each training package was delivered by floppy disk, CD-ROM and the Internet. It was not the intention of this research to compare the three media, but to investigate the acceptability of computer based training in its different modes and to explore its potential to ameliorate some of the difficulties faced by LAs in providing professional training for its health and safety enforcement staff. As Kearsley (1991) has pointed out, if different media are used to their full potential, they would not teach the same material and therefore could not be meaningfully compared. The significant determinant of a successful outcome to training is that the power of a particular medium is fully exploited when used.
Three subjects for which there was a recognised training need at the time of the research were selected as the basis of the training content for the packages, namely: The Health and Safety (Enforcing Authority) Regulations 1998, Hotel Safety and Accident Investigation. The aims and objectives for each training package were defined and matched against the suitability and strengths of the selected media (see Table I). Thus:
1) the Health and Safety (Enforcing Authority) Regulations 1998 training which was entirely textbased and sought to develop a systematic approach to the interpretation of the regulations and was delivered on floppy disk;
2) the hotel safety training which included a large amount of information, both textual and visual, sought to enable trainees to recognise hazards and assess compliance with legislation was delivered on CD-ROM;
3) the accident investigation training which incorporated a non linear approach (e.g. the ability to move between the different witness statements and factual details), reference to other on-line information sources, the facility to discuss professional judgments with fellow trainees and sought to develop accident investigation techniques was delivered via the internet.
All three packages were developed and produced inhouse at Salford University. The rationale for inhouse production was to ensure the training content was tailored to the needs of LA based enforcement officers and to enable the researchers to realize accurate data on the resources involved in developing training of this sort.
Table I: The suitability and strengths of the selected training media
Whilst the material covered by the floppy disk package could have been produced as a paper based exercise, the material presented via the CD-ROM and the Internet was highly visual and exploited the elements of these media (e.g. use of computer generated effects, hyperlinking to other useful information sources and the ability to communicate electronically with other trainees) and the information was therefore not easily transferable to a paper based exercise.
The aims and learning objectives of each package were defined and are presented in Table II.
Although acquisition of knowledge and skills is an important element of a training programme, the primary focus of this research was to examine the suitability of the differing media. Consequently, there was no attempt to report on any knowledge and/or skill gain following completion of the package. However, health and safety managers could introduce this aspect into future training experiences.
A sampling frame for selecting trainees was constructed to ensure that the three packages were tested by a range of individuals representing different employee status and different types of local authorities. Whilst every effort was made to ensure broad representation, allocation of particular packages was in some cases, limited by the availability of technology within individual LAs. The research tool randomly allocated trainees to particular packages without consideration of their preparedness to undertake the training package. Previous computing experience was not taken into account when trainees were allocated to packages because it was important to learn about their broad acceptability of these different media to a wide ability range amongst trainees.
Initially 95 trainees were recruited (32 to the disk; 33 to the CD-ROM; 30 to the Internet). The training packages were designed for delivery via computers located in the workplace. The managers of all the trainees were advised that the training should be completed at work and all agreed to this aspect. Each trainee was supplied with detailed user instructions and for the floppy disk and CD-ROM, the training disks; those using the Internet site were sent details of the web site address. In addition to completing the training package, each trainee was also asked to complete a pre and post training questionnaire and a user log. Trainees who did not complete all three items were excluded from the final analysis. Thus, a final sample of 59 trainees was obtained (14 disk, 25 CD-ROM and 20 Internet).
The pre-training questionnaire gathered personal details about the trainee, including their previous experience of computer-based technologies. The post-training questionnaire explored six aspects of the training (design and layout; training content, achievement of learning objectives; flexibility, usability and acceptability). In the user log, trainees were asked to give details of the frequency and length of time they spent using the package. No guidance as to the length of time each package should take to complete was given as this would, to some extent, be determined by a trainee’s professional experience, previous use of the technology and the different reasons (e.g. refresher training for experienced officers, introduction or review for newly qualified officers) for undertaking the training.
Table II: Aims and objectives of training packages
The use of a control group was inappropriate since it was not possible to construct a broadly similar work based learning package in the case of accident investigation and hotel safety. A paper based training package would not adequately handle the volume of information and the extent of the visual content in the case of hotel safety and there is no direct equivalent in a local authority workplace of access to on-line information sources or the facility to discuss with other geographically remote trainees as available for the webbased accident investigation training package.
The views of managers from twelve of the LAs who had participated in the pilots were also sought about the potential of the media to provide professional development training in the future.
The evaluation considered three key areas:
Pedagogical aspects; the capability of the media to deliver training which developed the desired learning objectives:
Ergonomic aspects; the reliability, usability and flexibility of the media. These factors may determine the acceptability of the media to trainees and managers and often tend to be the overriding factor in the selection of a particular medium:
Economic aspects; what can be afforded and/or justified in terms of cost.
The modest numbers involved in the trialling of each package, made significance testing inappropriate and in any case it was not the aim of the research to compare the relative acceptability of each of the three media. The results are therefore presented as percentages.
Pedagogy is the science of teaching and investigates the process of learning. Many factors influence the outcomes of training programmes (Patrick, 1992) and may include; the personnel involved; the nature of the skills to be acquired; the time that can be devoted to training; the media to be employed in facilitating learning; and the opportunities available to apply the knowledge and/or skills acquired. To evaluate the pedagogical impact, trainees were asked to consider three aspects of the training package they had tested and to attach a score (on a 1-10 scale) to each. The mean scores are presented in Table III.
Those trainees who had used the CD-ROM were the most positive overall about the pedagogical aspects of the medium, although the Internet group was also enthusiastic about its potential. The floppy disk package scored lowest on all pedagogical issues, possibly as a consequence of its limited functionality and visual attractiveness. Trainees and managers were asked more detailed questions on pedagogical issues and the results are presented below.
Design and Layout
The majority of the trainees (61%) reported that they found navigation through their package straightforward (Disk 57%; CD-ROM 60%; Internet 65%). Overall, 64% of trainees reported that the design and layout of the package maintained their interest, although there were large differences between the groups (Disk 29%; CD-ROM 84%; Internet 65%). The poor response of the disk users may be related to the limitations inherent in delivering training on floppy disk, in that only simple graphics can be incorporated. However it may also be linked to the different subject matter covered in the packages. The content of the floppy disk was likely to be less stimulating than that contained within the CD ROM and the web based training packages.
The majority of the trainees (76%) reported that they felt in control of their learning (Disk 71%; CD-ROM 92%; Internet 60%) and 66% reported that they were satisfied that they had completed their training, although differences did arise between the groups (Disk 71%; CD-ROM 88%; Internet 35%). The floppy disk and CD-ROM users were likely to have been more familiar with these forms of technology compared to the Internet users and this may have affected the learning experience of the Internet cohort as they reported little previous experience and understanding of web based technology. The lower scores reported by the Internet group for these two aspects may reflect the importance of providing greater guidance and/or training in moving around the site and in the tasks trainees are expected to complete within the Internet packages.
Table III: Impact scores for pedagogical aspects
Eighty three percent of the trainees reported that they found the training material interesting, although the disk trainees returned the lowest scores on this aspect (Disk 50%; CD-ROM 96%; Internet 90%). Overall, 27% of the trainees reported that the training was too simple for them, although this overall score was attributed mostly to the disk group (Disk 57%; CD-ROM 16%; Internet 20%).
Achievement of learning objectives
In order for trainees to have the potential to achieve the learning objectives set within the packages, two pre-requisites are necessary. Firstly, they must have the underpinning knowledge and skills pertinent to the subject matter covered by the training. Secondly they must have the technical know-how to operate the delivery medium. Both of these elements were considered in the research. Overall, the trainees reported high levels of appropriate subject knowledge and skills (81%) (Disk 86%; CD-ROM 88%; Internet 70%). However, whilst all of the disk cohort (100%) and 84% of the CD-ROM trainees felt that they had sufficient technical know-how to operate their training package, only 50% of the Internet cohort felt suitably prepared. The inexperience of the Internet cohort was also highlighted in that only three trainees reported exploring the hyperlinks within the site and only two reported visiting other web-sites whilst undertaking the training. The lack of experience in using the Internet may have important implications for the introduction of this medium for training. LAs wishing to exploit its potential will need to establish the previous Internet experience of their officers and may need to provide appropriate introductory courses to enhance their confidence in using this medium for training delivery. Nevertheless, managers must also be aware that once trainees become more skilled in the use of the web there may be a tendency for them to leave the controlled training environment to surf irrelevant sites. This aspect will need monitoring to prevent misuse.
Sixty-nine percent of the trainees reported that the learning objectives were clear, with the disk group returning the lowest scores on this aspect (Disk 57%; CD-ROM 80%; Internet 65%). Seventy-one percent of trainees reported that as a consequence of completing the training they felt more confident in undertaking the tasks covered by the package, although again, large differences arose between the groups, with the Internet trainees reporting the lowest levels of confidence (Disk 93%; CD-ROM 88%; Internet 35%).
Managers’ views on pedagogical aspects
Eleven of the twelve managers interviewed thought CBT was capable of developing technical knowledge and legal and procedural application, but half of those questioned did express some reservations about its ability to develop practical skills. In particular managers were sceptical about the ability of training provided on floppy disk to enhance practical skills and two commented that this form of delivery had few, if any advantages over paper based training. In contrast, CD-ROM and web-based training was thought to be capable of contributing to the development of practical skills if packages were designed to maximise interaction and were supplemented by other forms of skills training such as peer review and coaching.
The potential of CBT to promote consistency of enforcement because each trainee received the same information was noted as a clear advantage of CBT. However it was recognised that the information provided needed to be clear and comprehensive if the opportunities for misinterpretation are to be limited.
If training is to be delivered by computer, aspects such as the reliability, usability and flexibility of the hardware and software will be paramount (Ravet & Layte, 1997) and may determine whether such packages are widely adopted by LAs. With this in mind, trainees were asked to rate (on a 1-10 scale) the flexibility, usability and acceptability of the media they had tested. The mean scores are presented in Table IV.
Table IV: Impact scores for ergonomic aspects
As with the pedagogical aspects, the trainees commented favourably about the CD-ROM and were positive towards the Internet but less so towards a floppy disk program. Trainees and managers were asked more detailed questions relating to ergonomic aspects and these results are presented below.
New technologies have provided the opportunity to explore novel approaches to learning and to offer potential solutions to the challenges presented by the demands for more flexibility in the delivery of training in terms of time, location, content and form (Furst-Bowe, 1996). The vast majority (85%) of the trainees enjoyed being able to work at their own pace and this was particularly noticeable in the disk group where all trainees responded positively, compared with 84% of the CD-ROM trainees and 75% of the Internet trainees.
The complexity of the learning objectives varied between the packages and it was estimated that trainees would need approximately two hours to complete the disk package and six hours for the CD-ROM and Internet packages. Trainees were not directed about the expected completion times in order to learn more about the way in which trainees approached these packages. Fifty eight percent of the cohort did report that their normal work activities had prevented them from spending as much time as they would have liked on the package (Disk 36%; CD-ROM 72%; Internet 55%). Overall 56% of the trainees reported that they were able to work through the package without disturbance, but differences were apparent between the groups (Disk 71%; CD-ROM 64%; Internet 35%) and may be related to the amount of time required to complete the package as well as the individual experiences of the trainees. These findings raise important issues regarding the management of training. If employers move towards workbased learning they must ensure sufficient time is set aside for its completion and to facilitate reflection and the transfer of learning.
The overwhelming majority of the participants (86%) found the training medium they piloted easy to use, but differences did arise in the need for in-house support to run the programs with 14% of the disk group, 36% of the CD-ROM group and 75% of the Internet group requiring assistance. In general the programs operated without significant problems, but 44% of the CD-ROM group experienced some problems running their program, these were generally caused by local computer virus protection measures.
A prospective user’s overall attitude toward using a computer is a major determining factor in their eventual decision as to whether to use CBT. Reluctance to use CBT cannot be overcome merely by increasing a trainee’s exposure to this type of training. The user must gain in confidence, see the computer as beneficial to them, feel in control when using the computer and see the computer as an integral part of their work activities (Selwyn, 1997). The vast majority of trainees enjoyed using CBT (83%) (Disk 79%; CD-ROM 88%; Internet 80%) and 86% of the trainees expressed the wish to use the medium that they had piloted in future training programmes. Only 20% of the trainees reported that technical difficulties had discouraged them from using CBT with the Internet group reporting the highest rates (Disk 14%; CD-ROM 12%; Internet 35%).
Managers’ view on ergonomic aspects
The flexibility of CBT was seen as a distinct advantage by the majority of the managers (10 out of 12). Staff were able to access CBT at any time which afforded managers flexibility in terms of staff deployment. Training sessions could therefore be rescheduled without incurring cost should an emergency arise. The majority of managers also appreciated the advantages of staff being trained on site, as this reduced training and travelling costs and avoided the need for staff to be absent from the workplace.
Two managers who did not comment favourably about this aspect were employed in LAs in which the provision of information technology equipment was limited, thus negating the flexibility CBT can offer. Several of the managers also commented on the flexibility CBT provided in terms of how the training was used. For example, the training needs of different groups of inspectors can be addressed as elements of the training can be used by newly qualified officers whilst experienced officers can use the same package for refresher training.
There were some minor technical difficulties in the use of the packages and half of the managers reported that their staff had experienced problems with the computer hardware - in some cases this was attributed to computers not being of a sufficiently high specification and in others there were insufficient machines available for use in the workplace. Such obstacles may hinder LAs wishing to the use of embrace CBT as a training medium.
All but one manager thought their staff lacked the necessary computing skills to undertake the web based training and four believed their staff did not have sufficient computing skills to make the best use of the CD-ROM and the disk based training packages. There was a general view that staff needed to become more familiar with navigation of the web before they were able to use the Internet training to its full potential.
All twelve managers were prepared to use other CDROM and web-based packages if they were available. Two of the eight managers whose staff piloted the disk based training did not see any future for training delivered in this way because of its limitations compared to the other media piloted. Only one of the managers commented upon the relative advantage accruing because web based training is sourced from a single point i.e. the host server. Other forms of CBT such as the floppy disk and CD ROMs require some form of library system if maximum benefits are to be obtained for the purchasing organisation. The absence of comments from other managers about this aspect may reflect their inexperience in managing CBT.
Whilst training is important and necessary, it can also be costly and organisations must maximise the return on the investments they make in training their workforce (Read & Kleiner, 1995). Several research studies have concluded that, under the right circumstances, computer based delivery systems are considerably more cost effective than classroom teaching and produce learning that is at least equal, if not superior, to that which can be achieved in a lecture (Goldstein, 1993). Economic considerations may ultimately determine whether CBT is widely used within a training framework. The costs of each package (assuming technology and equipment to support it was already available in the workplace) was estimated, based upon the time taken to develop the training content and to write the supporting software divided by an estimated number of potential purchasers. It was also assumed that 40% of LAs would purchase the packages, resulting in a predicted cost of £100 for floppy disk, £400 for CD-ROM and £100 for the Internet site. The higher cost of the CDROM was due to the substantially greater software development time. The Managers were then asked to comment on whether they felt that, at these prices, the training packages represented an economically viable prospect. Five of the eight managers whose staff had used the floppy disk package did not believe that this type of technology had any long-term future. Managers may however have under-estimated the potential of disk-based training in providing an alternative to paper based training materials.
Opinions as to the economic viability of the CD-ROM package varied. Two believed it to be cost effective; a further four felt it would be in areas of recurring need for training, whilst three felt that the purchase cost would be too high. Thus, whilst the CD-ROM package was enthusiastically received by both managers and trainees, financial constraints within LAs may prevent its widespread adoption as a training medium.
All of the managers agreed that the Internet training package provided at this price would be cost effective, although one manager did raise concerns about the recurrent costs of working on-line. However, with recent developments in on-line pricing this is unlikely to be a future concern.
Discussion and Conclusions
Following evaluation of the packages developed within the research project it is clear that the use of CBT within LAs does provide an acceptable delivery mode for health and safety enforcement officer training. CBT provides an innovative and flexible medium through which theory and worked examples can be combined into workbased training packages which are trainee centred. The indications are that CBT has the potential to extend and augment the range of training methods presently employed.
Although the group piloting the Internet package did experience some difficulties in its use, both trainees and managers acknowledged the pedagogical and ergonomic potential of this medium to deliver training. Managers clearly viewed the Internet as the most economically viable means of addressing professional development of their health and safety enforcement staff in the future. Evaluation of the Internet package did however highlight the critical importance of managing the introduction and implementation of web-based training within LAs. Only 30% of the Internet trainees believed they had the necessary technical expertise and pre-requisite knowledge to undertake the training. Lack of familiarity with the medium was evidenced by their inability to exploit features such as hyperlinks and on-line discussion groups which were incorporated to facilitate learning. Inexperience with the Internet must therefore be addressed if its potential as a delivery method is to be realised. Its use may require more careful management compared to more familiar CBT media. Time may need to be ring-fenced to ensure trainees have an effective training experience and further consideration may need to be given to the design of the package to incorporate tasks that necessitate the use of features such as on-line searching and discussion groups. In addition to clarifying the tasks and milestones within the training, these activities will facilitate the development of navigation skills around the site.
Previous research has suggested that resource limitations and inappropriate content of courses are two of the key problems facing LAs (Prince et al., 1994). CBT offers a solution to these difficulties if LAs work collectively to commission and finance the development and production of web based training. The training content can be designed to address specific training requirements at a reasonable cost. Other advantages are that the input of experts can be harnessed; expert input is only required in the development stage and once captured it can be made available on a wide geographical scale and over a prolonged period of time. Financial savings are also realised because staff complete the training within the workplace at a time which is tailored to accommodate the demands of the employer, rather than a training provider. Managers commented that the provision of CBT enabling all enforcement officers throughout England, Wales and Scotland to receive the same training irrespective of location and time would make a major contribution to the achievement of consistency in enforcement. Assuming that the training content is up to date, CBT can be used repeatedly over a number of years ensuring all officers receive the same information. This is very different from the traditional forms of training in which the content varies depending upon the individual trainer’s delivery. Trainees may interpret the messages they receive in different ways and some of the managers pointed out that there was a need within CBT to include self assessment exercises that monitor the understanding and progress of trainees. The real issue is the way in which the training is transferred into the work setting and one of the key aims is to seek consistency in enforcement. It was not possible within the limitations of the project to examine the effects of the training on consistency of enforcement. However it is unlikely that there will be consistency in enforcement if officers are not provided with the same information in the first instance.
There are a number of pedagogical advantages in using CBT. It provides an opportunity for learners to adopt differing learning styles or strategies - an aspect which is not normally addressed in traditional lecturebased programmes (Patrick, 1992). Repeated access to the material provides an instant refresher course to those working in a particular area, or a general introduction to those who may be moving into a new area of work. The layering of material accommodates a range of abilities and prior knowledge and enables trainees to work through the material at their own pace. Engaging trainees through interaction with the medium can also support deep learning and encourage the trainee to validate knowledge through experimentation (Kolb, 1984).
Computer based training (CBT) is increasingly an integral element of professional development in the workplace and has the potential to improve the effectiveness of LA health and safety inspector training in a number of important ways. However, it is important to recognise that CBT introduces new demands on both the trainees and their employers (Robinson, et al., 1998). CBT moves control of the material covered to the trainee and therefore requires a more pro-active approach to learning. Managers must recognise a fundamental change in the way in which they organise training provision (Reeve et al., 1998). A shift towards training in the workplace requires them to organise and oversee the achievement of learning objectives as trainees progress through the training packages. If trainees and organisations are unclear about the management of this process, motivation to complete the training may diminish (Keeling et al., 1998). Failure to acknowledge the time and support needed to acquire new study skills may result in alienation (Lund & Volet, 1998). In order to promote successful implementation of CBT, empirical research that explores not only knowledge acquisition but also the impact of various contextual factors and personal characteristics on the effectiveness, productivity and success of CBT is essential to implementing health and safety enforcement officer training within Local Authorities.
The authors wish to thank the Local Authority participants who were involved in the planning and piloting of the CBT and the Local Authority Unit of the Health and Safety Executive which funded the research.
Cabinet Office (1999) Better Regulation Task Force – Enforcement Review. London, Central Office of Information.
Department of Trade and Industry (DTI) (1993) Review of the Implementation and Enforcement of EC law in the UK. London: HMSO.
Furst-Bowe, J. A. (1996) An Analysis of the Competencies Required by Trainers to Use Computer Based Technologies and Distance Learning Systems. Performance Improvement Quarterly, 09, 04, 57-78.
Goldstein, I. L. (1993) Training in Organizations (3rd Edition). California: Brooks/Cole Publishing Company.
Keeling, D., Jones, E. & Botterill, D. (1998) Work-Based Learning, Motivation and Employer-Employee Interaction: Implications for Life Long Learning. Innovations in Education and Training International, 35, 04, 282-291.
Kolb, D. A. (1984) Experiential Learning. New Jersey: Prentice Hall.
Lund, C. & Volet, S. (1998) Barriers to Studying Online for the First Time: Students’ Perceptions. EdTech’98 Proceedings. Available at web site: http://cleo.murdoch.edu.au/gen/aset/confs/edtech98/pubs/articles/1/lund.html
Patrick, J. (1992) Training: Research and Practice. London: Academic Press.
Prince, L., Campbell, A. and Nanton, P. (1994) Training in Health and Safety Enforcement for Local Authority Environmental Health Officers and Technicians. Institute of Local Government Studies.
Ravet, S. and Layte, M. (1997) Technology Based Training. London: Kogan Page.
Read, C. W. and Kleiner, B. H. (1996) Which Training Methods are Effective? Management Development Review, 09, 02, 24-29.
Reeve, F., Gallacher, J. & Mayes, T. (1998) Can New Technologies Remove Barriers To Work-Based Learning? Open Learning, 13, 03, 18-26.
Robinson H, Smith M, Galpin, F. et al. (1998) As Good As IT Gets: Have We Reached the Limits of What Technology Can Do for Us? ALT-C 98 discussion workshops. Proceeding published in Active Learning 9, 50 –53.
Selwyn N. (1997) Teaching Information Technology to the ‘Computer Shy’ A Theoretical Perspective On A Practical Problem. Journal of Vocational Education and Training, 49, 03, 395-408.