This article is available in: HTML PDF jgi: p. 1

Journal Information
Journal ID (publisher-id): jgi
ISSN: 1910-7595
Publisher: Centre for Addiction and Mental Health
Article Information
The Centre for Addiction and Mental Health
Received Day: 30 Month: May Year: 2011
Accepted Day: 24 Month: January Year: 2012
Publication date: December 2013
First Page: 1 Last Page: 11
Publisher Id: jgi.2013.28.9
DOI: 10.4309/jgi.2013.28.9

Evaluation of the quality of services provided by a gambling helpline: an empirical study
Francine Ferland1
Robert Ladouceur2
Isabelle Giroux2
Cathy Savard2
Sophie Ruel2
Marie-Frédérique Leclerc3
François Guimond2
1Service de recherché CRDQ/CRDCA, Centre de réadaptation en dépendence de Québec, Québec (Qc), Canada
2École de psychologie, Université Laval, Québec (Qc) Canada
3Faculté de médecine, Université Laval, Québec (Qc) Canada
This article was peer-reviewed. All URLs were available at the time of submission.

For correspondence: Francine Ferland, Ph.D., Service de recherché CRDQ/CRDCA, Centre de réadaptation en dépendence de Québec, Québec (Qc), Canada; E-mail: francine.ferland@ssss.gouv.qc.ca
Competing interests: None declared (all authors).
Ethics approval: The project was approved by the ethical committee for the research with human subjects of Laval University (« Comité d'éthique de la recherche sur des êtres humains de l'Université Laval »); the project was approved under the title « Évaluation des impacts de la ligne téléphonique « Jeu : aide et référence » et de service qui y sont attachés ». The approval number is 2008-209.
Funding: The research project was funded by the Fonds de recherche : Société et culture – Québec.
Contributors: Francine Ferland, working in collaboration with Professors Robert Ladouceur and Isabelle Giroux, conceptualized the study and created the method used to assess all its identified goals. Professor Ferland also ran the analysis, worked with the other authors on the interpretation of the data, and wrote the final draft of the paper.
Robert Ladouceur, working with Professors Ferland and Giroux, helped conceptualize the study and identified the most effective method to assess all goals. Professor Ladouceur also contributed to the study through appropriate revision of the final drafts of the paper, and through provision of his gambling research expertise, thereby improving the scientific value of the research.
Isabelle Giroux, in collaboration with Professors Ferland and Ladouceur, helped conceptualize the study and contributed to the writing of the first drafts of the paper. Professor Giroux also assisted in the interpretation of the results.
Cathy Savard worked with Professor Ferland. She coordinated the study, supervised data collection and data entry, and contributed to the writing of the earlier drafts of the paper.
Francine Ferland, Ph.D., is a psychologist and the coordinator of the Service de recherché CRDQ/CRDCA in Quebec City (Canada). She is responsible for the research service, conducts research projects with clients in treatment at the Service, and collaborates with academic researchers interested in substance addiction and pathological gambling. Besides her regular duties at the rehabilitation center, Dr. Ferland is also associate professor at Université Laval (Quebec) and Université de Sherbrooke (Quebec), where her responsibilities include the co-direction of master's and Ph.D theses. She is also a regular researcher for the Institute on Addictions, which operates under the auspices of the Centre de réadaptation en dépendence de Montréal.
Robert Ladouceur, Ph.D., is professor emeritus at the Université Laval (Québec) and a researcher in the field of gambling. He has published over 450 scientific papers, and has received numerous awards throughout his career. He is well-known for his cognitive-behavioural method in the treatment of pathological gambling.
Isabelle Giroux, Ph.D., is an assistant professor at the École de Psychologie of Université Laval and the Director of the Centre québécois d'excellence pour la prévention et le traitement du jeu (Quebec Centre of Excellence for the Prevention and Treatment of Gambling). Dr. Giroux has been working in the field of gambling research for 18 years. Her research interests include fundamental aspects of gambling such as the perception of randomness and treatment programs for at-risk and pathological gamblers.
Cathy Savard, Ph.D. is a clinical psychologist working in the domain of addiction. Her thesis addressed gambling and problem gambling among youth.
Sophie Ruel, Marie-Frédérique Leclerc and François Guimond, working in collaboration with the other researchers, ran the bogus phone calls and performed the data entry. They also contributed substantially to the interpretation of the data before writing the first versions of the paper.
Sophie Ruel, B.A., completed her bachelor degree in psychology and is currently writing her Ph.D. dissertation in health psychology, specializing in cancer counselling.
Marie-Frédérique Leclerc, B.A., completed her bachelor's degree in psychology and is currently a student in medicine.
François Guimond has completed a bachelor degree in psychology and has a specialiazation in psychoeducation.

Abstract

This study evaluates the quality of the services provided by a helpline according to the type of caller and the type of request. One hundred and fifteen bogus calls made by simulated clients were used for the evaluation. Results showed that 87% of the calls received an adequate response from the helpline information/referral counsellors (IRCs). Referral requests made by a loved one were less frequently rated as adequate. The quality of responses was better for requests for a referral than they were for information. The simulated client method allowed for experimentation in how a caller could respond emotionally when contacting the helpline. The method revealed that answers given to a loved one could be frustrating and that some IRCs have more difficulties providing general information about gambling than providing the name of a rehabilitation centre. These results should help to improve the quality of the services offered by the helpline to respond more specifically to the callers' information and referral requests.

Résumé

La présente étude évalue la qualité des services fournis par un service d'assistance téléphonique selon le type d'appelant et le type de demande. L'évaluation a porté sur cent quinze faux appels de personnes se faisant passer pour des clients. Les résultats ont montré que dans 87 % des appels, les appelants ont obtenu une réponse adéquate des conseillers et conseillères en information et aiguillage. Les réponses aux demandes d'aiguillage faites par des proches ont été moins souvent classées comme adéquates. La qualité des réponses était meilleure pour les demandes d'aiguillage qu'elle ne l'était pour les demandes d'information. Utiliser des personnes se faisant passer pour des clients a permis d'expérimenter les diverses réactions émotionnelles possibles d'un appelant qui utilise le service d'assistance téléphonique. Cette méthode a révélé que les réponses données à un proche pouvaient être frustrantes et que certains conseillers et conseillères avaient plus de difficulté à fournir des renseignements généraux sur les jeux de hasard et d'argent qu'à donner le nom d'un centre de réadaptation. Ces résultats devraient aider le service d'assistance téléphonique à améliorer la qualité de ses services pour répondre plus adéquatement aux demandes d'information et d'aiguillage des appelants.


Introduction

The last 20 years have been marked by a decrease in taboos associated with gambling and an increase in services providing help to problem gamblers and their families and friends. Following in the footsteps of several other countries, Canada and its provinces implemented telephone helpline services available all year and around-the-clock. Although the respective mandates of the various helplines differ, most of them offer support, information about gambling problems, and referrals. Unfortunately, the quality of the services offered has rarely been empirically evaluated. This present study works to fill this gap by using a simulated client method to evaluate the quality of the services provided.

Since 1993, the province of Quebec (Canada) has made available a free, anonymous and confidential helpline for people with a gambling problem. This helpline offers support, information and referrals to gamblers who are concerned about their gambling habits, as well as to their family and friends (Jeu : Aide et Référence, 2008). The line receives over ten thousand calls per year, and most of them (59%) are requests for referrals (examples including rehabilitation, self-help groups and counselling) or for general information about problem gambling (34%) (Jeu : Aide et Référence, 2008).

The helpline is being used mostly by people who are in despair. Therefore it is important that the services offered be adequate and pertinent to the needs of those persons. To date, few empirical studies have evaluated the quality of these services. Although some information has been provided by helpline employees, only two formal evaluations have been conducted by independent researchers unconnected to the helpline organization. Chevalier, Papineau, and Geoffrion (2003) conducted the first study, which revealed that the services offered by the helpline failed to meet gamblers' expectations, who requested either counselling or someone with whom to discuss their problems. Callers were often disappointed that they were commonly only given a telephone number to contact. The second study assessed the needs and expectations of gamblers who used the helpline (SOM recherches et sondages, 2005). The results of this study indicated that the callers believed they were listened to, respected, and supported during the call itself, but were also occasionally irritated when they could only reach an answering machine.

Although these studies provided interesting information about the helpline and its services, none of them evaluated the quality of the services. Gilbert and Sutton (2006) mentioned that the difficulty in including a control group in helpline satisfaction studies consequently limits the generalization of the results. Clifford (2008) added that it is also difficult to conduct a randomized study in such an evaluation.

The simulated client method might be the best way to solve these problems. This technique has been used in several fields – including health, financial services and retail sales – to evaluate the quality of services provided to customers (Bos, Visser, Tempert, & Schaalma, 2004; Reubsaet, Lechner, & De Vries, 2006; Wilson, 2001). This method consists of placing an observer in a fictitious role simulating an ordinary customer. It allows researchers to collect pertinent information concerning the quality of the service, without requiring the intrusive presence of a third party (Huntington & Schuler, 1993). According to Madden, Quick, Ross-Degnan, and Kaffle (1997), the simulated client method is practical, easily adaptable to different contexts and reasonably cost effective.

In the context of evaluating the quality of services provided by a gamblers' helpline, the simulated client gathers information as if he or she were a real caller seeking a treatment center (e.g., by looking for information about problem gambling, or being in need of someone to talk to about a family member's or one's own gambling problem). This study verifies the adequacy of the helpline information/referral counsellors' (IRCs) intervention according to the type of request (referral, information, support) and the type of caller (gambler, relative, counsellor). Considering that the initial mandate of the helpline is to offer information, provide referrals and guide gamblers to appropriate resources, it is postulated that the quality and the adequacy of the intervention will vary according to the type of request.


Method
Bogus Telephone Calls

One hundred and fifteen (N  =  115) bogus calls were made to the telephone helpline between January 26 and March 24, 2009. Of these phone calls, 49% were requests for referral, 42% were requests for information and 9% were requests for someone to talk to or for support. The calls were placed every day of the week and were spread out over various times of day. More precisely, 49 calls were made during the daytime (between 7 a.m. and 3 p.m.), 55 during the evening (between 3 p.m. and 11 p.m.) and 11 calls were made during the night (between 11 p.m. and 7 a.m.). This distribution of the bogus calls corresponds to the breakdown of calls normally received by the telephone helpline (Jeu : Aide et Référence, 2008).

Instruments

Scenarios. Eight different scenarios representing three types of calls and three categories of callers were used. Three of the eight scenarios were referral request scenarios (from a gambler, a counselor, and from family or friends, respectively), three were information request scenarios (again, from a gambler, a counselor, and from family or friends) and two scenarios requesting a person with whom to speak or for support (one from a gambler, and one from family or friends). No calls simulating a person with suicidal thoughts were used in this study.

Technical grid. A technical grid was used to assess the context and the characteristics of each call. It comprised 13 questions documenting the date, time, scenario used, changes made to the scenario, number of attempts made before reaching the IRC, wait time prior to a response, duration of the telephone call, gender of the IRC, and problems encountered during the call.

Integrity of the scenarios. This form was used to ensure that the main elements of the scenario were explored. It also assessed the uniformity of the scenarios among the three experimenters. All the items in this list were classified using a dichotomous scoring procedure. Adequacy was positive if all expected responses were positive and negative if at least one of the expected responses was negative.

IRC's evaluation grid. This grid was developed according to the evaluation tool used to evaluate new IRCs. It had three scales: aptitude and attitude of the IRC, knowledge of the IRC, and intervention skills of the IRC. The aptitude and attitudes section comprised six items relating to such concerns as attention and concentration. The knowledge section included five items and the intervention skills section contained eight items, including the ability to be empathic and to relate to the person calling. All items were graded using a 5-point Likert scale, where 1 corresponds to “Deficient” and 5 corresponds to “Very good”. Note that all instruments used in this study are available upon request from the main author.

Procedure. The managers of the helpline and the helpline IRCs were informed, prior to the beginning of the project, about the start and end dates of the period during which the bogus calls would be made. The managers were also informed that the calls would be recorded. Neither the specific time of the calls nor the scenarios of the calls were revealed to the managers and the IRCs. The IRCs were also advised that the recordings would be treated with strict confidentiality, would not be given to any helpline managers, and that all the recordings would be destroyed at the end of the project. At the end of the study, each IRC completed a consent form either refusing or allowing the researchers to use the calls which involved them. All IRCs agreed that their recorded calls could be used.

The bogus calls were carried out by three undergraduate students in psychology trained in applying each experimental scenario. The objective of this training was to make the calls as realistic and standardized as possible. For each call, the experimenter blocked his telephone “call display” function, varied the area from which he or she was calling, and modified the voice in order to reduce the chance of being recognized by the helpline IRCs.

Interjudges' agreement. Interjudges' agreement was agreed upon in 38 of the bogus calls (33%). A total of 722 items were rated by the three experimenters and 85% of them were rated similarly by all experimenters upon a first listening. Discussions among the experimenters allowed for the standardization of the rating process.


Results
General Characteristics of the Calls

The calls lasted an average of 6.85 minutes (SD  =  3.96). Although the experimenters received an immediate response for 69% of the calls, 27% of the calls went without an immediate answer and had to be re-tried before they were answered, 1.7% required two attempts and 2.6% required more than two attempts. It is important to note that 22% of the calls were put on hold by the IRC. Among these calls, one was switched over to another IRC after having been put on hold. Another call was switched over to another IRC once the IRC was presented with the scenario.

Adequacy of the Responses Provided by the Helpline IRCs

An adequacy score was calculated for the response provided to each call. Eighty-seven percent (87%) of the calls had a positive score and 13% had a negative one. The chi-square test done on the percentage of adequate responses showed a significant difference between the three types of requests, χ2 (2, N  =  114)  =  7.29, p < 0.05. The Pearson standard residual analysis indicated that the percentage of adequate responses was significantly lower for calls requesting information than for calls requesting support or referrals.

The analyses did not show any difference in the percentage of adequate responses between the three types of callers when the call was a request for information, χ2 (2, N  =  48)  =  5.41, p > 0.05. However, a significant difference was found among the types of callers when the caller requested a referral, χ2 (2, N  =  56)  =  10.77, p < 0.01. The Pearson standard residual analysis indicated that the percentage of adequate responses was significantly lower for calls from family or friends than for calls from gamblers or counsellors. It should also be noted that meaningful statistic differences could not be determined for support requests because all calls in this category were judged adequate. Table 1 shows the percentage of adequate response for all three request types.

Quality of the IRCs' response

A nested variance analysis was carried out on the total score obtained for each scale as well as for the score obtained for each item. The variable “Type of caller” was nested into the variable “Type of request,” which made it possible to evaluate the effect of the “Type of caller” within the three types of requests. When a significant difference was found, a contrast analysis (Bonferroni-Holm method) was performed to identify the main cause.

Scale 1: Helpline IRCs' aptitude and attitudes. The analyses carried out on the total score showed no significant difference for the “Type of request” and the nested effects. Analysis of the items indicated that only one item showed a significant difference (“The helpline IRCs provide a response that satisfies the caller's expectations”). The analysis of variance indicated that the “Type of request” effect was significant (F (2, 106)  =  5.68, p < 0.01) whereas the nested effect was not, F (5, 106)  =  1.52, p > 0.05. The contrast analysis revealed that the average score attributed to the referral requests for this item was significantly higher than the average score attributed to the information requests (4.79 vs. 4.20; t (106)  =  3.36, p < 0.01).

Scale 2: Helpline IRCs' knowledge. Total score was not calculated for this scale due to the large variance in the number of calls that applied to each item. Moreover, the analyses per item did not reveal any significant effect.

Scale 3: Helpline IRCs' listening skills. The analyses carried out on the total score for this scale did not show any significant difference (note: the item “Objectivity of the helpline IRC concerning the treatment facilities” was not included in the total score because it only applied to 98 out of 115 calls). However, the analyses carried out on each item indicated that only the item “Ability to identify the need of the caller” showed a significant difference for the “Type of request” (F (2, 106)  =  4.35, p < 0.05) whereas the nested effect was not significant, F (5, 106)  =  1.52, p > 0.05. The contrast analysis carried out on this item revealed that the average score attributed to the referral requests was significantly higher than the one attributed to the information requests (4.84 vs. 4.36; t (106)  =  2.95, p < 0.01).


Discussion

The goal of this study was to evaluate the quality of the services provided to callers using the Quebec gambling helpline. The primary objective was to verify differences in the adequacy of the helpline intervention for differing types of callers with varying types of requests. Results revealed that a significantly lower number of adequate responses were provided for referral requests when the call was made by family or friends than when the request for referral was made by a gambler or a counsellor. When the scenario involved a referral made by family or friends, the interview was scripted so that the caller would ask for the contact information of a treatment center, with the claim that the caller could pass it on to someone in the family who was suffering from a gambling problem. Bogus calls revealed that helpline IRCs were less likely to provide such referral information to a third party than directly to the gambler. Such a response is undoubtedly upsetting for a loved one who is seeking appropriate resources to help a family member suffering from a gambling problem.

The second objective was to verify whether the quality and the adequacy of the intervention differed according to the type of request made to the helpline (referral, information, support). The hypothesis, which stated that adequacy and quality of the intervention would differ according to the type of request, was confirmed. The percentage of adequate responses was significantly lower for calls requesting information than for calls requesting support or referral. However, responses for information requests varied from one IRC to another and certain IRCs seemed less comfortable responding to this type of request than did others. This observation was previously reported by Chevalier et al. (2003), which stated that callers looking for general information instead of contact information were often troubled by the response.

Results pertaining to the quality of the intervention also revealed that the ability to identify the needs of the caller and the ability to provide a response that meets the caller's expectations were higher for calls requesting a referral than for those requesting information. This discrepancy might be explained by the fact that it is easier for the helpline IRCs to identify the needs of the caller when the caller requests a referral directly rather than indirectly. It is also possible that the IRCs' training focuses more on the resources available to treat pathological gamblers as opposed to offering more general information about theories related to gambling problems, thus creating a positive bias towards the referral request.

This study has at least two limitations. The first limitation concerns the estimation regarding whether a helpline IRC's answer was in fact adequate. In this study, adequacy of the responses was determined through the requests included in the various scenarios and not through the types of responses normally provided by the helpline IRCs for the same requests. A second limitation concerns two aspects of the method. The large number of calls (115) were made over a short period of time (two months) and the number of helpline IRCs taking the calls was small. Despite having used various strategies to attempt to limit the possibility that the helpline IRCs might recognize the experimenters, it is still possible that they were recognized nonetheless, and this determination may accordingly have influenced the IRCs' responses.

Despite these limitations, these findings are important because they clearly reflect the services offered by the gambling helpline. The study is also one of the first to examine the quality of the services offered by a helpline using an empirical evaluation. The simulated client procedure appears to be an excellent way to obtain non-biased information and to account for the actual experiences of callers. This method highlights some difficulties related to service accessibility, particularly placing callers on hold during a call. Taking into consideration that pathological gambling often has significant consequences and effects on both family and friends as well as on the gambler (Ladouceur, Sylvain, Boutin, & Doucet, 2000), it can be frustrating for a caller not to reach a live person on the other end of the line (Chevalier et al., 2003) or to be put on hold by the helpline IRC. The overall results of this study should contribute significantly to improving the services that helplines offer to their respective callers.


References
Bos, A. E. R.. , Visser, G. C.. , Tempert, B. F.. , & Schaalma, H. P. . ( 2004). Evaluation of the Dutch AIDS information helpline an investigation of information needs and satisfaction of callers.. Patient Education and Counseling, 54, 201–206.
doi:10.1016/S0738-3991(03)00214-3
Chevalier, S.. , Papineau, É.. , & Geoffrion, C. . ( 2003). Évaluation du programme expérimental sur le jeu pathologique. Rapport 9. Les lignes téléphoniques dédiées aux jeux de hasard et d'argent [Evaluation of the pathological gambling experimental program. Report #9. Helplines dedicated to gambling]. Institut national de santé publique du Québec, Montréal: AU.
Clifford, G. . ( 2008). The evolution of problem gambling helplines. In Zangeneh, M. , Blaszczynski, A. , & Turner, N. E.
(Eds.), In the pursuit of winning: Problem gambling theory, research and treatment (pp. 291–312). New York, London: Springer.
Gilbert, H.. & Sutton, S.. ( 2006). Evaluating the effectiveness of proactive telephone counselling for smoking cessation in a randomized controlled trial. Addiction, 101, 590–598.
doi:10.1111/j.1360-0443.2006.01398.x
Huntington, D.. & Schuler, S. R.. ( 1993). The simulated client method Evaluating clientprovider interactions in family planning clinics.. Studies in Family Planning, 24, 187–193.
Jeu: Aide et Référence ( 2008). Rapport annuel de jeu : Aide et référence : 2007–2008 [Annual Report on Gambling : Help and Referral, 2007–2008]. Montréal: AU.
Ladouceur, R.. , Sylvain, C.. , Boutin, C.. , & Doucet, C. . ( 2000). Le jeu excessif Comprendre et vaincre le gambling Excessive gambling Understanding and treating gambling problems..
Les éditions l'Homme, Montréal.
Madden, J. M.. , Quick, J. D.. , Ross-Degnan, D.. , & Kafle, K. K. . ( 1997). Undercover careseekers Simulated clients in the study of health provider behavior in developing countries.. Social Science & Medicine, 45, 1465–1482.
Reubsaet, A.. , Lechner, L.. , & De Vries, H. . ( 2006). The Dutch cancer information helpline More critical patients after 10 years.. Patient Education and Counseling, 63, 215–222.
doi: 10.1016/j.pec.2005.10.011
SOM recherches et sondages ( 2005). Étude qualitative portant sur les besoins et les attentes des joueurs à l'endroit des services téléphoniques d'aide au jeu [Qualitative evaluation of the gamblers' needs and expectations about gambling helplines]. Ministère de la Santé et des Services sociaux du Québec, Montréal: AU.
Wilson, A. M. . ( 2001). Mystery shopping Using deception to measure service performance.. Psychology & Marketing, 18, 721–734.
doi: 10.1002/mar.1027

Tables
Table 1 

Adequacy of The Responses to Scenarios According to The Type of Request.


Click for larger image


Article Categories:
  • Research

Related Article(s):


Copyright © 2020 | Centre for Addiction and Mental Health
Editor-in-chief: Nigel E. Turner, Ph.D.
Managing Editor: Vivien Rekkas, Ph.D. (contact)