Research

Did Inclusion of Illinois Bias the Medicare Chiropractic Services Demonstration?

James M Whedon, DC

 

author email    corresponding author email   

Topics in Integrative Health Care 2010, Vol. 1(2)   ID: 1.2004



Published on
December 30, 2010
Text Size:   (-) Decrease the text size for the main body of this article    (+) Increase the text size for the main body of this article
Share:  Add to TwitterAdd to DiggAdd to del.icio.usAdd to FacebookAdd to GoogleAdd to LinkedInAdd to MixxAdd to MySpaceAdd to NewsvineAdd to RedditAdd to StumbleUponAdd to Yahoo

ABSTRACT

Purpose: The Medicare Chiropractic Services Demonstration was conducted in ninety-three counties from 2005 to 2007.  An analysis of budget neutrality (ABN) found that the demonstration was associated with increased total Medicare expenditure of $50 million among chiropractic users.  Demonstration sites in Illinois accounted for 98% of the increase in Medicare expenditures among chiropractic users.  This study explored the association between chiropractic utilization and total expenditure per beneficiary in general to generate a hypothesis to explain how inclusion of Illinois counties biased the results of the ABN.

Methods: This analysis explored the association between demonstration effects and total concurrent costs to Medicare in all beneficiaries (contextual effects).  Medicare data were analyzed to generate descriptive statistics for the ninety-three demonstration counties and matched comparison samples before, during and after the time of the demonstration.  Expenditures per beneficiary (EPB) were determined and variation in EPB was quantified.  Contextual effects were compared with the demonstration effects.

Results: EPB for all Medicare beneficiaries was consistently higher in Illinois than in other states, and did not vary substantially during the demonstration years.  Of the ten sites with the highest EPB in 2006, nine were in Illinois.  In Illinois EPB was 228-$373 higher during the demonstration years.  There was a congruence between contextual effects and demonstration effects in Illinois that was not observed in the other four demonstration states.

Conclusions: An association between chiropractic costs and all costs in Illinois (not observed in other sites) may have confounded the reported effect of the demonstration in Illinois.

INTRODUCTION

The Medicare Chiropractic Services Demonstration was conducted in 93 US counties from April 1, 2005 through March 31, 2007.  The demonstration sites included all sixteen and thirty-three counties in Maine and New Mexico respectively, seventeen counties in Virginia, twenty-six counties across northern Illinois that included the Chicago area, and one adjacent county in Iowa.  The purpose of the demonstration was to determine the effects of expanded coverage for chiropractic services under Medicare.  As a part of the official evaluation of the demonstration, an analysis of budget neutrality (ABN) was performed to determine the effect of the demonstration on Medicare costs.1  The Centers for Medicare and Medicaid Services (CMS) did not limit the demonstration to evaluation of the direct cost of expanded chiropractic services because CMS hypothesized that expanded chiropractic coverage might affect other costs to Medicare.  The ABN measured total Medicare expenditure in two groups: 1) beneficiaries with NMS diagnoses who were chiropractic users, and 2) all beneficiaries with NMS diagnoses.  Medicare expenditures in demonstration counties were compared with expenditures in primary comparison samples, which were created by combining data from two sets of matched control countiesThe ABN found that the demonstration was associated with increased Medicare expenditure. [Table 1]  Demonstration sites in Illinois accounted for $49 million of the total $50 million increase in Medicare expenditure in chiropractic users, and 80% of the increase in Illinois was attributed to the city of Chicago and its suburbs.1

Table 1. Excerpted Results of the ABN* 1

 

Chiropractic Users

Patients with NMS Conditions

Increased expenditure for chiropractic services

$35 million

$35 million

Increased expenditure for non-chiropractic services

$15 million

$79 million

Total

$50 million

$114 million


*Analysis of Budget Neutrality, Medicare Chiropractic Services Demonstration

It is unknown whether or not the inclusion of Illinois counties as demonstration sites biased the results of the ABN.  If inclusion of Illinois counties resulted in a sampling of counties that was nationally unrepresentative with regard to the outcome of interest, such a systematic error would have biased the ABN and rendered its results invalid. Unanswered questions about the utilization and expenditure environment within which the demonstration was conducted hinder an evaluation of bias in the ABN.  These questions include:

1.  Is there an association between expenditures for chiropractic users and total expenditures under Medicare?

2.  If such an association exists, is it causal, and in what direction?

2.  How does chiropractic utilization and expenditure vary geographically?


This study was intended as a preliminary exploration of the first question.  Medicare data were analyzed for trends in overall Medicare expenditures, and compared with trends reported by the ABN.  This study analyzed mean annual expenditures per beneficiary (EPB) for demonstration and comparison sites from 2003 – 2008.  The aim of this analysis was to explore Medicare expenditures reported by the ABN within the context of total concurrent costs to Medicare in all beneficiaries.  The goal was to hypothesize how inclusion of Illinois counties among the demonstration sites may have biased the ABN.

METHODS

This study employed a retrospective serial cross-sectional design.  Medicare fee-for-service data were analyzed to generate descriptive statistics for the ninety-three demonstration counties and their matched comparison counties - before, during and after the time period of the demonstration.  Two data sources were used: 1) data on the ninety-three demonstration counties and matched comparison counties, and 2) county-level payment and denominator data on Medicare beneficiaries 65 years of age and older.  The two files were merged to generate separate annual files for each year analyzed (2003-2008).  Data for matched comparison counties were combined to form a single comparison sample for each demonstration county.  All six annual files were then merged on the county code variable into a single analytic file that contained the number of Medicare beneficiaries and total Medicare Part A and B fee for service payments for each of the 93 demonstration sites and matched comparison samples.  The data files were merged using STATA 11 (StataCorp LP, College Station, TX).

These data were analyzed for mean expenditures per beneficiary (EPB) for demonstration sites and matched comparison samples, by year and by state.  Variations in EPB were quantified with extremal ratios (ER) and coefficients of variation (COV).  Differences in mean EPB between demonstration sites and matched comparison samples (contextual effects) were compared, both longitudinally and geographically, with differences in mean EPB in the populations evaluated by the ABN (demonstration effects). Because the contextual data were drawn from100% of the Medicare beneficiary population, the geographic and temporal differences reported in the contextual data were not inferred, but actual, so no significance testing was performed.  No significance testing was performed on demonstration data because only the summary statistics were available to the author.  Analyses were performed and charts generated using STATA 11 and Microsoft EXCEL 2007 (Microsoft Corporation, Redmond, WA).

RESULTS

Mean EPB for all Medicare beneficiaries in demonstration sites trended higher from 2003 to 2008, ranging from $4,907 in New Mexico in 2003 to $8,336 in Illinois in 2008.  EPB for all Medicare beneficiaries was consistently higher in Illinois than in other states throughout the time period studied. [Figure 1]  The amount of variation in EPB for all Medicare beneficiaries did not change substantially during the demonstration years (2005: ER 2.34, COV 0.14; 2006: ER 2.14, COV 0.13; 2007: ER 2.13, COV 0.14). Evaluation of the distribution of demonstration site EPB for all beneficiaries in 2006 (the middle year of the demonstration) demonstrated the greatest variation in New Mexico extremal ratio 1.77) and Illinois (extremal ratio 1.65); of the ten sites with the highest EPB in 2006, nine were in Illinois. [Figure 2]

Figure 1. Contextual Effects: Mean EXP in Illinois Sites vs Other Sites, 2003-2008

  Contextual Effects: Mean EXP in Illinois Sites vs Other Sites, 2003-2008


EXP =Expenditure per Beneficary, IA = Iowa,  ME = Maine, NM = New Mexico, VA = Virginia


Figure 2. Distribution of Mean Total Medicare Expenditure per Beneficiary in Demonstration Sites, by State, 2006

  Distribution of Mean Total Medicare Expenditure per Beneficiary in Demonstration Sites, by State, 2006


Each dot represents one of the 93 demonstration counties.
IA = Iowa,  IL = Illinois, ME = Maine, NM = New Mexico, VA = Virginia


Contextual effects were generally lower relative to comparison samples in Iowa, Maine, New Mexico and Virginia; a small increase in Iowa in 2006 was the only exception.  In Illinois the contextual effect was a consistently higher EPB throughout the time period ($228-$373 higher during the demonstration years). [Table 2]  Analysis for longitudinal variation in contextual effects demonstrated that the direction of overall contextual effects observed from 2003 to 2008 was opposite that of demonstration effects. [Figure 3]  While EPB for all beneficiaries in demonstration sites trended lower than comparison samples before, during and after the demonstration, the ABN found that EPB for chiropractic users and patients with NMS diagnoses was higher in demonstration sites.

Table 2. Comparison of Demonstration Effects with Contextual Effects

 

Illinois

Iowa

Maine

New Mexico

Virginia

Demo Effect - Chiropractic Users*

$485

-$178

$35

-$59

$136

Demo Effect - NMS Patients**

$213

-$148

-$109

-$110

-$78

Contextual Effect, by Year***

 
 
 
 
 

2003

$294

-$45

-$513

-$1,559

-$972

2004

$328

-$262

-$396

-$1,558

-$1,138

2005

$292

-$275

-$481

-$1,611

-$395

2006

$373

$40

-$544

-$1,632

-$1,495

2007

$228

-$148

-$570

-$1,446

-$1,669

2008

$54

-$56

-$516

-$1,584

-$1,532


*[Mean expenditure per beneficiary (EPB) for chiropractic users in demonstration  site state] minus [Mean EPB for chiropractic users in matched comparison sample], April 2005 – March 2007 1
**[Mean EPB for patients with neuromusculoskeleatal diagnoses in demonstration  site state] minus [Mean EPB for patients with neuromusculoskeleatal diagnoses in matched comparison sample], April 2005 – March 2007 1
***[Mean expenditure per beneficiary for all beneficiaries in demonstration site state] minus [Mean EPB for all beneficiaries in matched comparison sample], 2003-2008


Figure 3. Juxtaposition of Demonstration Effects with Contextual Effects, 2003-2008

  Juxtaposition of Demonstration Effects with Contextual Effects, 2003-2008


*[Mean expenditure per beneficiary (EPB) for chiropractic users in demonstration  site state] minus [Mean EPB for chiropractic users in matched comparison sample], April 2005 – March 2007 1
**[Mean EPB for patients with neuromusculoskeleatal diagnoses in demonstration  site state] minus [Mean EPB for patients with neuromusculoskeleatal diagnoses in matched comparison sample], April 2005 – March 2007 1
***[Mean expenditure per beneficiary for all beneficiaries in demonstration site state] minus [Mean EPB for all beneficiaries in matched comparison sample], 2003-2008


In contrast to the longitudinal analysis, analysis for geographic variation in contextual effects demonstrated some congruence between demonstration effects and contextual effects. [Figure 4]  Juxtaposition of the contextual effects observed for 2006 (the middle year of the demonstration period) with reported demonstration effects, stratified by state, facilitates comparison of size and direction of effect.  Congruence between contextual effects and demonstration effects appeared to be higher in Illinois than in the other four demonstration states.

Figure 4. Juxtaposition of Demonstration Effects with 2006 Contextual Effects, by State

  Juxtaposition of Demonstration Effects with 2006 Contextual Effects, by State


*[Mean expenditure per beneficiary (EPB) for chiropractic users in demonstration  site state] minus [Mean EPB for chiropractic users in matched comparison sample], April 2005 – March 2007 1
**[Mean EPB for patients with neuromusculoskeleatal diagnoses in demonstration  site state] minus [Mean EPB for patients with neuromusculoskeleatal diagnoses in matched comparison sample], April 2005 – March 2007 1
***[Mean expenditure per beneficiary for all beneficiaries in demonstration site state] minus [Mean EPB for all beneficiaries in matched comparison sample], 2006


DISCUSSION

Variation in Total Expenditure per Beneficiary



The results demonstrate that EPB for all beneficiaries trended higher in Illinois than in the other demonstration states before, during and after the time period of the demonstration.  Because relative levels of EPB by county tend to persist over time,2 the high rates of EPB described by the ABN for Illinois sites may not have been confined to the time period of the demonstration.  The ABN results in Illinois may have been tied in part to a trend of increased EPB for all beneficiaries.  An analysis of nationwide variation in Medicare EPB at the county level from 2002-2005 characterized Chicago area counties as “high cost”.2  Areas characterized as “low cost” included much of New Mexico and parts of Virginia.  High cost and low cost counties were those that were ranked in the highest and lowest quintiles respectively, after controlling for observable differences.  Thus, the choice of demonstration sites included counties from both extremes of the EPB spectrum, contrary to one of the stated goals of the demonstration site selection process.3  No county-level normative data are available for comparison of the results of the variation analysis.  Nationwide, the COV in state-level EPB in 2005 was 0.11 and declining,4 so the COV of 0.13-0.14 reported here for EPB in all beneficiaries in the demonstration sites is probably higher than the national average.

Comparison of Contextual Effects with Demonstration Effects



The analysis of contextual effects by year clearly demonstrated that the demonstration effects reported by the ABN ran contrary to overall longitudinal trends in the same Medicare environment.  However, analysis of contextual effects by state (in 2006, the middle year of the demonstration) revealed a somewhat more nuanced picture.  The contextual effect was generally lower, except in Illinois, where EPB in demonstration sites was higher relative to comparison samples.  The one exception was Iowa, where the contextual effect was slightly higher.  However, the single Iowa county included in the demonstration was contiguous with the Illinois demonstration counties, suggesting a clustering effect in common with the high expenditure sites in Illinois.

Review of the demonstration effect for Illinois illustrates that in both the chiropractic user analysis (n = 155,086), and the NMS analysis (n=1,049,963) of the ABN, the effect was significantly increased EPB, while the mean effect for Iowa, Maine, New Mexico and Virginia combined was decreased EPB.1  Similarly, the contextual effect in Illinois was increased EPB, and the mean effect in the other demonstration states was lower.  It is likely that the effect reported by the ABN in Illinois reflects the tidal effect of overall trends in Medicare expenditure, and is not solely attributable to the demonstration.  Indeed, because the cost of chiropractic services constitutes less than 0.5 percent of all Medicare costs, effect-cause is a more plausible explanation for the demonstration “effects” reported for Illinois.  Effect-cause as an explanation for the demonstration results in Illinois is made more likely by the apparent congruence, not observable in other states, between demonstration effects and contextual effects.  Comparison of demonstration and contextual effects reveals striking incongruence in effect size in Maine, New Mexico and Virginia, and some opposing effect direction in Iowa, Maine and Virginia.  In Illinois, however, all three measures of effect were congruent with regard to direction, and greater congruence was apparent with regard to effect size.

It is clear that Illinois demonstration sites had higher EPB (both absolutely and relative to matched comparison samples) in chiropractic users, NMS patients, and all beneficiaries.  Illinois sites were utilization and expenditure outliers relative to the other demonstration sites.  Nationally, the Chicago hospital referral region occupied the 90th percentile in overall Medicare spending in 2006.5  Although higher EPB in Illinois do not make the sample of demonstration counties unrepresentative, congruence in Illinois between the demonstration effects and the overall utilization and expenditure environment, which was not observed in other states, suggests a clustering of high utilization that crosses health discipline boundaries.  Geographic variations must be interpreted within context.6  Regarding geographic variations in utilization, it has been observed that clustering of physician behavior is likely tied to the formation of physician preferences that are formed within and in response to a local culture.”7 It is likely that the results reported by the ABN in Illinois were not solely an effect of the demonstration, but rather were tied to supply-sensitive care decisions consistent with a local culture of relatively high utilization.

LIMITATIONS

The ABN was a complex and exhaustive component of the evaluation of the Medicare Chiropractic Services Demonstration, and this study only touches upon selected parts of ABN methodology and results.  A complete review and re-analysis of the ABN is beyond the scope of this study.  This analysis was not intended to directly re-evaluate the findings of the ABN, but rather to examine certain findings of the ABN in the light of contextual data on expenditures for all Medicare beneficiaries. The Medicare expenditure data reported here are for all Medicare beneficiaries and are not restricted, as were the ABN data, to beneficiaries with NMS diagnoses.  Data on Medicare expenditures were collected and analyzed by calendar year, but demonstration data were collected from 4/1/05 to 3/31/07, so direct comparisons of annualized data were not possible in this study.  Calendar year 2006 data were juxtaposed with demonstration data for purposes of comparing contextual effects with demonstration effects.

With regard to the use of the word, “effect”: no causal effect was established by the ABN.  As noted by Stason et al, “These costs may be effects of the demonstration or may be unrelated to it.” 1  The report’s conclusion ignored the import of this caveat and implied a causal effect in stating, “Both the all NMS user and chiropractic user analyses conclude that the demonstration increased Medicare payments…” 1  The ABN referred to demonstration “effects” throughout, as in the statement, “The overall demonstration effect was to increase Medicare payments by $114 million.”1  Although used inappropriately in the ABN, the word “effect” is, for purposes of consistency and brevity, also used throughout this paper to refer to EPB statistics associated with the time period of the demonstration.  It should be noted that neither the results of the ABN nor of this observational study should be construed as effects in the true sense, as in the result of a cause-effect relationship.

CONCLUSION

The cost increases reported to be associated with the Medicare Chiropractic Services Demonstration were likely caused not solely by the demonstration, but by a local provider culture in Illinois that favored increased utilization for all Medicare beneficiaries.  It is hypothesized that inclusion of Illinois counties among the demonstration sites resulted in a sampling of counties that was nationally unrepresentative with regard to the outcome of interest.  The bias resulting from this systematic error may have rendered invalid the results of the ABN.  A study of geographic variations in Medicare expenditures for chiropractic, and associated variations in expenditure for non-chiropractic services could reveal whether or not the inclusion of Illinois counties led to the evaluation of an unrepresentative sample population by the ABN.

ACKNOWLEDGEMENTS

This study was funded by a grant from the National Center for Complementary and Alternative Medicine (NCCAM) of the National Institutes of Health, and The Bernard Osher Foundation.  The views expressed herein do not necessarily reflect those of NCCAM or the NIH.  The author derives income from the part-time practice of chiropractic health care, but has no other conflicts of interest to disclose.
Share:  Add to TwitterAdd to DiggAdd to del.icio.usAdd to FacebookAdd to GoogleAdd to LinkedInAdd to MixxAdd to MySpaceAdd to NewsvineAdd to RedditAdd to StumbleUponAdd to Yahoo

References

1.   

Stason WB, Ritter G, Shepard DS, et al. Final Report: Evaluation of the Demonstration of Expanded Coverage of Chiropractic Services under Medicare. Waltham, MA: Brandeis University; May 12, 2010.



2.   

Hopson A, Rettenmaier A. Medicare Spending Across the Map. Dallas, Texas: National Center for Policy Analysis; 2008.



3.   

Livermore GA, Stapleton DC. Medicare Chiropractic Services Demonstration Final Design Report. Washington, DC: Cornell University Institute for Policy Research; April 27, 2005.



4.   

Congressional Budget Office. Geographic Variation in Health Care Spending. Washington, DC: 2008.



5.   

The Dartmouth Atlas of Healthcare. Medicare Reimbursements per Enrollee, by Race and Program Component: Chicago, IL. The Dartmouth Atlas of Healthcare  Accessed November 27, 2010.

 [ Full-Text Link ]

6.   

Greenland S. Ecologic versus individual-level sources of bias in ecologic estimates of contextual health effects. Int J Epidemiol. Dec 2001;30(6):1343-1350.



7.   

Evans RG. There's no Reason For It, It's Just Our Policy. Healthcare Policy. 2009;5(2):14-24.