About the Data

Introduction

The data on this website are based on responses to a voluntary, statewide survey of patients who received care at healthcare practices in Maine. The most recent survey was completed over a four-month period in late 2014 and early 2015 by randomly selected adult patients and parents of children receiving care in one of 336 participating primary care, pediatric and specialty care practices. This is the second statewide survey of patients’ experiences in Maine, and provides important information from the patients’ perspective about the care they receive. The first survey was conducted in late 2012 and early 2013.

The survey is sponsored by the Maine Quality Forum (MQF). MQF required practices to use standardized survey instruments that have been scientifically tested and developed by the federal Agency for Healthcare Research and Quality (AHRQ). The surveys – the Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) and the Patient Centered Medical Homes CAHPS (CAHPS PCMH) 2.0 surveys – ask patients about aspects of their healthcare experience that are closely linked to quality, such as access to care and the strength of the doctor-patient relationship.

How can this data be used?

Knowing how patients feel about their experience from visiting a healthcare practice can help patients compare practices across the State, choose practices that are best for them, and help practices identify areas where they can improve.

For example, if a practice scores low in communicating with patients about how to take care of their own health, patients may want to ask for more information when they visit that practice, or the practice could provide patients with more advice and guidance on these issues as part of the regular visits.

Survey Participants, Collection and Question

Who was surveyed and when?

A random sample of Mainer’s who had at least one visit to a particicipating practice site in 2014 were surveyed:

  • Adult patients (18 years of age or older) of participating primary care and specialty care practice sites.
  • Parents of children (under 18 years of age) who are patients of participating pediatric practice sites.

Surveys were sent to patients between August 2014 and February 2015 and asked about the care they received in the last six months or 12 months, depending on which survey was used.

How were the surveys collected?

Participating practices provided a list of all their patients who visited the practice in 2014 to one of three survey vendors designated by the MQF.

Based on the number of full-time providers in the practice, a target number of completed surveys was established that were needed to make the practice results statistically reliable. Survey vendors randomly selected patients to be surveyed, assuming a response rate that would allow the practice to meet its targeted number of completed surveys. Surveys were sent by mail with two mail follow-ups to non-respondents. All survey responses are anonymous; providers in the practices do not know which patients were surveyed. Surveys were returned to the vendor, and the vendor entered the data for the practice into the National CAHPS Database. Neither the practice nor the MQF has access to individual surveys completed under this initiative.

What types of questions were patients asked?

Questions included on the CG-CAHPS and the CAHPS PCMH surveys are grouped and summarized into categories that are most important to patients, including:

  • Getting Timely Appointments, Care and Information – Five questions asked patients about how often they got appointments for care as soon as needed, got timely answers to questions when they called the office, and saw the doctor within 15 minutes of appointment time.
  • How Well Providers (Doctors) Communicate with Patients – Six questions asked patients how often their doctors explained things clearly, listened carefully, showed respect, provided easy-to-understand instructions, knew their medical history, and spent enough time with the patient.
  • Helpful, Courteous and Respectful Office Staff – Two questions asked patients how often office staff were helpful and treated them with courtesy and respect.
  • Attention to Your Mental or Emotional Health (adults only) – Three questions asked patients whether the provider talked to them about worry or stress in their life, feeling sad or depressed, or other personal or family problems.
  • Talking with You About Taking Care of Your Own (or your child’s) Health (child only) – Two questions asked patients or, for children, their parents about whether the provider worked with them to set specific goals for their/their child’s health and asked about things that make it hard for them to take care of their child’s health.
  • Talking About Medication Decisions (adults only/PCMH CAHPS only) – Three questions asked patients whether the provider talked about reasons to take or not take a medicine and asked the patient what they thought was best for them regarding the medication.
  • Doctor’s Attention to Your Child’s Growth and Development (child only) Six questions asked parents whether the doctor talked about their child’s growth, learning abilities, age-appropriate behaviors, moods and emotions, screen time and ability to learn and get along with others.
  • Doctor’s Advice on Keeping Your Child Safe and Healthy (child only) – Five questions asked parents whether the doctor talked about and gave printed handouts or booklets about keeping their child from getting injured, the food the child eats, physical activity, and household problems.
  • Patients’ Rating of the Providers – One question asked patients to rate their doctor in that practice on a scale of 1 (worst provider possible) to 10 (best provider possible).
  • Other Individual Questions: Access to Care, Information About Care and Appointments, and Attention to Care from Other Provider Questions – In addition to the questions above, one of the two surveys that were allowed – the CAHPS PCMH survey – asks seven stand-alone individual questions that were not combined into categories. These questions were newly added in the CAHPS PCMH survey and are not on the CG-CAHPS survey. They were not included in the categories because they were not included in prior years and would result in unfair comparisions over time.

These questions asked patients about whether they got care they needed after hours or got information about what to do if they needed care on evenings, weekends or holidays; how many days they had to wait for urgent care; whether they received reminders between visits; whether the provider’s office followed up to give them results for blood tests, x-rays or other tests; whether the provider talked with them about all their prescriptions; and seemed informed and up-to-date about care they got from specialists. These questions are only available if the practice used the CAHPS PCMH survey. For practices, that used the CG-CAHPS survey, you will see “Not Available” indicating that this item is not available for this survey.

The website presents summary scores for each category and, when show details is clicked, scores for the questions that make up the category appear. Ratings for questions that are not included in a category are shown individually.

Interpreting the Data

How are the survey results calculated?

The survey results reflect responses from 53,000 randomly selected patients who received care at participating medical practice sites/clinics. Scores from each patient at a practice site were combined into a single score for a practice site. Scores for each question were adjusted by the National CAHPS Database based on nationally established criteria for differences in patient age, education level, and self-reported general health status based on patient survey responses to these questions across practice sites.

What does the practice site score mean?

The practice site score is based on the percent of patients that reported the best possible answer to individual or combined questions. This is sometimes referred to as the “top box” score. For questions asking about the frequency that a provider did something, the score shows the percent of patients that said it “always” happened. For example, for getting appointments, the bar graph shows the percent of patients reporting that they always get appointments when they need it. For questions that asked whether something happened (Yes/No) the bar graph shows the percent that responded “Yes.” For questions that ask for an overall rating, the bar graph shows the percent that gave a rating of 9 or 10. If you roll-over a percent on the data display, text will describe the top box score for that specific question (e.g. percent of patients responding “always” or percent of patients responding “Yes”).

How does my practice site compare with other practices in the state?

Because the MQF used standardized national surveys and practices agreed to have their data publicly reported, we are able to show comparable results from all practices in Maine that participated in the MQF initiative whose data was submitted by their survey vendor into the National CAHPS Database.

The blue line on each bar graph represents the statewide average and shows how your practice compares to others in the state. Note: The statewide average represents the average for practices that participated in the MQF initiative and used the CG-CAHPS or CAHPS PCMH 6 or 12 month survey and that reported their survey results in the National CAHPS Database. The average does not include all practices in the state.

What does “Too Few Patients Responding” next to my practice mean?

If you see “Too Few Patients Responding” next to the name of your practice, it means that the practice volunteered to participate in the survey but that too few patients responded to an individual question or to the entire survey to achieve the minimum threshold for meaningful reporting (0.7 reliability level). Generally, practices had to have a minimum of 30 completed surveys to report a score for a category. In these cases, despite the good intention of the practice to participate, not enough patients who received surveys in the mail completed them.

What does “Not Applicable” on certain questions mean?

The MQF allowed practices to use either the CG-CAHPS or the CAHPS PCMH 6 or 12 month surveys. While both surveys ask many of the same combined questions, the CAHPS PCMH also includes some individual questions that are not asked on the CG-CAHPS survey. If you see “Not Applicable” next to your practice site on these individual questions, it means your practice used the CG-CAHPS survey and patients were not asked these questions.

Can I see results from last year’s survey?

The website has been updated to show the most recent survey results from 2014. If the practice participated in the MQF survey in 2012, you will see a calendar icon with the text, “Two Years of Results.” You can view those results by selecting 2012 Survey Results from the “View” drop-down at the top of the data display and the practice’s profile, if applicable.

Finding Data for your Specific Practice

What if I don’t know the name of my doctor’s office?

If you don’t know the name of your doctor’s office or practice site, the website includes a list of all doctors in each of the participating practices. In “Our Survey Results” in the navigation, you can search by practice name, medical group or provider’s name. You can either begin searching for a name and the results will appear below, or you can scroll entire list. Note: While you can search by provider name, the survey results are for a practice, not individual doctors within the practice.

What if my doctor’s office is not shown?

If you do not find your physician or practice site on the list, your practice declined to participate in this initiative. Practice participation was voluntary and required practices to use one of the the standard surveys required by the MQF and to agree to have survey results publicly reported. A total of 336 primary care and specialty care practices in Maine participated. Practices that participated are to be commended for being leaders in assessing their patients’ experience in order to improve the quality of care provided.

Getting More Detailed Results

How do I get more detailed survey results?

If you are a provider participating in the MQF’s patient experience survey initiative, you can find more detailed results from your practice’s surveys on the National CAHPS database. Consumers or employer groups interested in additional survey details than what are reported on Maine Patient Experience Matters, can download the full data on the Resources page. Additional details include the number of completed surveys per survey item and the percent of patients’ responding for each survey response category (in addition to top box score that is reported on the website). All of the information is presented in aggregate across patients reporting in the practice and does not include any specific patient information.

To determine if differences in the scores between a practice and the statewide average are reliably different and not just the result of chance, we did statistical tests to see when a practice site’s results were significantly “at,” “above” or “below” the statewide percent of patients reporting the most positive response statewide. Statistical significance (measured at p<.05) means you can be reasonably certain that the percentage differences are not just random or based on chance. Results of statistical significance testing are shown on the Excel spreadsheet that is available for download here, which also includes a tab that further explains how the data is organized and how to interpret results.

What can a practice do to improve patient experience scores, especially in areas that are lower than the statewide average?

To help practices identify the best strategies for improving patient experience, Maine Quality Forum and its partners at the Muskie School of Public Service at the University of Southern Maine developed a quality improvement toolkit that summarizes successful strategies from the literature that other organizations have used to address patient experience concerns.

View the Quality Improvement Toolkit online >