But there can be some risks involved in collecting data online.

Online surveys (hosted on platforms like SmartSurvey and SurveyMonkey) can provide researchers with multiple benefits such as:

• Access to large amounts of people (surveys links can be distributed widely)
• Cost efficient (some survey software is even free to use)
• Provides participants with anonymity and confidentiality, often leading to more honest and free responses (Teitcher et al., 2015).

However, if preventative steps are not taken, collecting data online can open up the opportunity for fraudsters to submit fake responses. A study by Bauermeister et al. (2012) has identified several patterns of fraud in online research:

1. Eligible participants who take studies twice, usually without malicious intent;
2. Eligible participants who take studies multiple times, in order to receive compensation (e.g. money or vouchers);
3. Ineligible individuals who fabricate responses, once or multiple times (often via bots) in order to gain compensation.

What is a bot?

Bots are internet software programs that are designed to imitate or replace human behaviour, typically undertaking repetitive tasks at a much faster rate that human users could perform. Often bots will be designed to complete surveys en masse.

What effect can this have on my research?

Duplicate and fake responses can not only compromise the quality and validity of research data but can also affect research budgets if fraudulent answers are not picked up on before any compensation is sent (Teitcher et al., 2015).

For example; if there was only enough money to recruit 50 participants, and two responses were found to be fraudulent, then the researcher is not only out of pocket but also only able to now recruit 48 participants. This is especially concerning if research is being conducted with funding from charities in which money is extremely tight.

How can you prevent fraudulent activity in online survey platforms?

• CAPTCHA. This test is designed to determine if an online user is really human and not a bot that might be programmed to take a survey multiple times. You have probable seen these on internet sites, but may not know it stands for ‘Completely Automated Public Turing Test to tell Computers and Humans Apart’. Many online survey platforms will allow you to add one of these as a question type. Add to the first page in your study to fend off bots.
• Prevent multiple responses on the same IP address. Many surveys will allow you to check a button that means participants cannot take the survey more than once on the same computer.
• Back Button. In ‘Survey Options’ you can enable a Back Button, which is useful for participants to be able to change their responses after consideration. However, this could potentially enable them to re-take the survey multiple times. Think carefully about the pro’s and con’s of adding or removing a back button.
• Change the order of questions. In most survey platforms you can set the questions to appear in a random order, this method can detect bots as it indicates a level of human attention. This may not always be possible, depending on your methodology.
• Make invite only and add a password: Instead of distributing an anonymous link on study advertisements, make the survey ‘Invite Only’ and request that potential participants email you for password access. This adds another layer of protection, as you can ask participants where they saw the study advertised and check their language, grammar and email address for fraudulent red flags.
• Consent form: Add a line about fraud to your consent form, such as, “I understand that I will not be compensated if my answers are suspected of fraudulent behaviour”. This may deter potential fraudsters, and will also cover you if they try to claim compensation.
• Pay respondents nothing, less, or provide a lottery: De-incentivise fraud by reducing the amount of compensation available. However, this may reduce uptake of the survey.
• Include an interview stage to study: This is not always possible or appropriate. However, including an interview stage to the study may deter fraudsters from participating and adds another means to detect fabricated responses.

How can you detect fraudulent activity in online survey data?

• Screen data for inconsistent responses: This is easier to do with qualitative research, as you can check that the answers are coherent, relevant and written in the relevant language. With quantitative research, check for inconsistent answers. Make sure to confirm fraudulent answers with other researchers/supervisors.
• Speed checks: You should have some idea of how long it would realistically take for a human to take your survey. Some software (such as Qualtrics) will show you this estimate. When checking answers, keep a close eye on any responses that took drastically less time to complete, or substantially more.
• Look for strange email addresses: Fraudsters will often create batches of email addresses with random letters and numbers preceding ‘@xxx.com’.
• If unsure, send an email to participant: Once you have checked with others, and are still unsure about potential fraud, you could send an email to the participant explaining the situation and request more information about where they saw the advert etc.

All of these preventative and reactive steps have their pros and cons, and not all will be possible due to methodological and ethical implications.

How can we help?

Evolving Communities can help you set up and run your online surveys, ensuring that they are protected from any bots or fraudulent activity.

They can also check data for fraudulent responses. Contact the team at info@evolvingcommunities.co.uk for more details.

Bauermeister, J. A., Pingel, E., Zimmerman, M., Couper, M., Carballo-Dieguez, A., & Strecher, V. J. (2012). Data quality in HIV/AIDS web-based surveys: Handling invalid and suspicious data. Field methods, 24(3), 272-291.  Link.

Teitcher, J. E., Bockting, W. O., Bauermeister, J. A., Hoefer, C. J., Miner, M. H., & Klitzman, R. L. (2015). Detecting, preventing, and responding to “fraudsters” in internet research: ethics and tradeoffs. The Journal of Law, Medicine & Ethics, 43(1), 116-133.  Link.

One way to access information to help you in your work is by using data from official government statistics.  Here, we look at the top five reasons why you should use statistics in your profession.

1.  Saves you time

You are a busy professional with a never-ending to-do list. You need statistics at your fingertips from a reliable source. Using a list of statistics from an official body such as the UK Statistics Authority saves you hours of searching around a number of different sources.

2.  It’s regulated

You don’t always know how reliable statistics are if they come from think tanks or political bodies.  Are they skewed by political agendas?  The Office for Statistics Regulation is a government organisation that assesses official statistics for compliance with the Code of Practice for Statistics.  They have to remain neutral.

3.  Helps you create useful policy based on need

You can’t create useful strategy and policy if you don’t have access to reliable statistics.  If you don’t know what the current state of play is, how do you create effective policy that aims to improve services or provision of care?  Looking at current information will help to inform your decision-making.

4.  Adds credibility

If you’re a researcher putting in for a grant, statistics are vital to the credibility of your application – you need to show that there is a need for the work you would like funding for.

5.  Improves evaluation

Statistics on local issues, services or topics can be used to evaluate whether an intervention has worked or not.  As well as using information to fill gaps, it can be used as a baseline to measure improvement in the future.  For example, health trusts could look at the figures for the number of people who self-harm.  Have those figures gone down because of the intervention they have put in place?  Has wellbeing improved since a specific service was introduced?

Dr Sara Nelson, Head of Research & Insight at Evolving Communities, said: “The use of statistics can help a wide of variety of people in their professions and the benefits will vary depending on the work involved.

“Commissioners from councils or Clinical Commissioning Groups will use statistics to create policy and strategy for their local areas.  If they are putting together a mental health strategy, they will need to know various related statistics for their area and more on a national level.  Do they have more people who say they suffer from an illness from the national average?  If yes, why is this and what can be done to deal with it?

“Charities may also use statistics in their campaign literature to help raise funds for their cause to show to supporters or grant-makers that their service is needed.”

Thoughts often turn to hours spent in rooms with no natural light, staring at endless PowerPoint slides and trying not to let your thoughts drift onto other more exciting topics.

It doesn’t have to be like that!  Evolving Communities hold contracts for three local Healthwatch organisations in Dorset, Gloucestershire and Somerset.  The teams and their dedicated, trained volunteers go out and engage with people regularly to gather their experiences of accessing health and care services and do a fantastic job.  But even they can be filled with trepidation when asked to carry out focus groups.

So, to take the terror out of conducting focus groups, Evolving Communities’ Research and Insight (R&I) team came to help them overcome their fear.  The R&I team have a background in academic research and evaluation and have experience in running small local focus groups as well as larger, more interactive workshops.

What did we do?

We brought together all our Healthwatch teams and some of their volunteers and Evolving Communities volunteers in a lovely old building, with views across the fields, at Lackham College in Wiltshire.  A pleasing venue immediately makes everyone feel more relaxed and ready to work!

We mixed everyone up so they weren’t sitting with their own teams, an opportunity to work and interact with new people.  There were a few PowerPoint slides to start but the majority of time was spent in interactive workshops.

We introduced a typical focus group scenario for each table to play out and everyone took on a different persona – a chance for a bit of acting! We even ‘planted’ some more challenging participants to test out the facilitators.

Everyone really got into character and took their acting role very seriously.  There was a lot of laughter and banter throughout the session.  A long way from those long days spent in PowerPoint purgatory.

At the end of the session we looked at the challenges encountered by the groups and discussed the best ways to overcome them in the future.  Following a break for lunch on the patio, we finished off the session with some tips and tricks for analysing and reporting the focus group data.

All in all, it was a positive experience, new links were made and everyone felt more confident in their ability to facilitate a focus group.

Learning can be fun.  Evolving Communities can help you and your staff to engage more effectively.  If you would like some training for your organisation in this area,please get in touch.

One way to overcome this is to employ alternative methods to the same old, rigid techniques that are used time and time again.  It is time to try something different! Here are five creative methods that we have or would love to make use of.

1. Go on the road!

Are you finding it difficult to get the public to come to you?  Then visit local people in their own communities!  During September 2017 Healthwatch Wiltshire embarked on a two-week Campervan & Comments Tour of the county in a 1969 VW Campervan called Milo.  The aim of the tour was to get the word out about Healthwatch, and to ask people to share their experiences of health and social care.  The tour visited supermarkets, schools, markets and leisure centres.  Free goodies were available to participants, and the press were invited along to cover the events.  Over the two-week tour 2,077 comments were collected!  The event was such a success, Milo will be on the road again in March 2019, this time visiting Somerset and Gloucestershire with the local Healthwatch teams.

2. Hold a themed community event

As mentioned in our previous blog post, community events can give you valuable insight into your research projects.  But why not push the boat out even further and theme your event around a fete or festival?  The Eden Project utilised this pioneering public engagement technique in order to attract a broad audience of people who would usually be apprehensive about ‘serious’ public consultations.  Around 1,000 people attended their community fete events, encouraged by the jovial atmosphere, fun activities and creative installations – elaborately decorated with heaps of bunting!

3. Participant filmmakers

The collection of research data using videos has become more common in recent years, but it can be time consuming, and may not capture the full experiences of the people it’s trying to represent.  One way to overcome this is to hand over the responsibility and creativity to the participants!  And that is exactly what Canadian researchers did in 2012 when working with youth from a remote Inuit community in Labrador.  Workshops were held in which young people were taught how to use video editing software and were tasked with creating a short video about their life experiences.  You can view the video here.

4. Digital storytelling

If you have a Facebook or Instagram profile, you have probably seen the page Humans of New York.  The Humans of New York photo-blog was started by photographer Brandon Stanton in 2010.  This form of digital storytelling involves interviews with strangers, who share their stories through portraits and mini-narratives.  Each story acts as a call to action and has the potential to increase engagement and interest.  This type of engagement method could work well with participants who only have limited amount of time to tell their stories, such as staff and patients in a hospital setting.  A series about childhood cancer was produced by Humans of New York.  Completed stories can be disseminated on social media or at a special ‘gallery’ style event.

5. Jazz up an interview with cultural probes

There’s a reason why interviews and focus groups are the most common tools for gathering feedback – they work really well.  But there is no reason why they can’t be enhanced and supported with creative add-ons.  Cultural probes have been used since 1999 and involve the use of tools, artefacts and tasks intended to incite people to think about their world in new ways.  It is particularly useful for hard to reach groups whose culture, behaviours and attitudes may not be well known to the researcher.  Kits are made up of things such as disposable cameras, maps, stickers, diaries, notebooks and postcards.  Activities should be fun, but professional.  Kits are delivered to the participants with instructions, and once completed the researcher meets up with participants individually, or in a group, to share their completed kits and engage in deeper conversation.  Key to analysis is heading back to the participant to check if the researchers have interpreted their findings correctly.

Get the right fit

As with any research, you will need to match your engagement methods to the community you wish to reach and the questions you want answered.  One of the best ways of doing this is to get your participants involved in the research process right from the start.  And for some, more traditional methods may still be king.

If you have used any creative engagement methods, why not send us a tweet?

Gloucestershire County Council are working with Leeds Beckett University on a UK wide programme to develop practical ways for local authorities to address obesity via a ‘whole systems’ approach.

The council wanted to gather the views of residents in Podsmead about their food choices and options. Podsmead was chosen as it has few shops and poor public transport, limiting access to affordable healthy food.

It’s personal

Food and eating is personal and can be a sensitive subject, especially if people feel criticised or blamed for their choices.  Early conversations with residents and staff at the local community café – the Melting Pot – were important in deciding what approach to take, what questions to ask and when and how to ask them.

Initial conversations with residents revealed that food should be about enjoyment and happiness and that it can offer comfort.  The Manager of the Melting Pot Café Josie said she saw food as a way of bringing people together – so the idea of a free community meal to promote the recently refurbished café was born.

Corn beef anyone?

The project team designed three separate questionnaires on shopping, cooking and good food to support conversations with residents at the meal, which included delicious Caribbean fare such as corn beef and rice, sweetcorn, home-made coleslaw and mango smoothies.

A team of young volunteers completed a shorter survey with their family and friends.  The questions were worded to avoid being intrusive and to enable comfortable conversations.

This ‘light touch’ approach helped deliver a successful community event.

Here’s what we learnt:

1. Language is important
‘Healthy eating’ can imply failure or judgement and may be impossible for people who are struggling.

2. People are wary of being seen as a problem to be fixed
Supporting existing community activity and strengths offers a ‘way in’ and gives something back.

3. Involve people as partners
Residents were involved as partners in designing and carrying out this research.  This was important to ensure that we asked relevant, sensitive questions that made sense to local people in the context of their lives.

4. Be willing to adapt your methods
Based on early feedback, we changed the name of the project from the ‘Healthy Eating Project’ to the ‘Food & Families’ project to make it more appealing to local residents.

5. Ensure you have sufficient funds
Working with people as partners needs funding – the free meal was a good incentive but extra funding may have encouraged more residents to get involved.

6. Involve other local organisations
Communities have a great deal of charities, groups and organisations who have great contacts and insights.  We worked closely with Healthwatch Gloucestershire and other local groups who had a great deal of local knowledge.

After the event, one of the volunteer researchers said: “It was a brilliant idea to bring people together and get ideas for making an even better community.

“People were quite open minded in giving their views and I think it helped that we were all getting together as residents, so they didn’t feel invaded.”

This is great to hear!  It was a short-term, small scale project but residents were clear and consistent in their feedback.  Our findings were supported by similar research and by local organisations, which enabled us to be confident in our recommendations to the council.

Read the full project report