Is the Agency Changing Course on CSA ? Not Likely
Report on MCSAC Meeting – June 18, 2013
In earlier emails, we have called your attention to a series of articles published in CCJ which are critical of SMS methodology and one which recently acknowledged our lawsuit and the importance of the publication of SMS percentile rankings in the vicarious liability/negligent selection debate. We posed for discussion the question of whether the agency was changing its policies in anticipation of oral argument, by suggesting that its future rulemaking on safety fitness determinations would be based upon objective rather than relative percentile rankings. We also took note of recent FMCSA announcements indicating that it would change its CSA website this summer, supposedly to clarify that SMS scores are not safety ratings and should not be viewed as such.
On Tuesday, June 18, 2013 we attended the Parker Committee’s meeting with the FMCSA to hear the presentation of the agency’s plans for the SMS website. This is our report.
First, a few interesting factoids were presented.
1. 48 million user inquiries to the SMS website have been logged during the past 12 months.
2. The number of active carriers the agency now claims to regulate is 525,000.
3. The agency has some data on 40% of the active carriers which in turn are involved in 80% of the recorded crashes.
4. Because carriers are not ranked until there are 3 driver inspections or 4 vehicle inspections, many carriers with data are unranked. The agency agreed that no conclusion can be drawn concerning the status of unranked carriers.
5. Currently, crash data reflects only “carrier involvement” rather than carrier causation, and accordingly is not made public.
Future Plans
The stated objectives of the agency are to:
1. Reinforce SMS as a prioritization of carriers, not as a safety rating!
2. Reinforce the agency’s safety mission by improving safety performance and getting bad actors off the road.
3. Focus efforts on formulating a new safety fitness determination which will free safety ratings from the necessity for investigations!
The agency still has under development its safety fitness determination criteria, which it hopes to release in a Notice of Proposed Rulemaking in January 2014. That rulemaking apparently will involve some objective criteria and not be based upon relative scoring or percentile rankings.
The Future of Percentile Rankings, Golden Triangles
and Website Publication of Scores
It was somewhat heartening to hear that the agency does not propose to use comparative safety scores to make its safety fitness determination of who is fit to operate on the nation’s highways. Yet, it is clear the agency is not abandoning the publication of percentile rankings, scoring mechanisms and the vicarious liability mischief which results.
Apparently the agency is not willing to concede that SMS methodology has an unfair branding effect on safe carriers, heightens vicarious liability concerns and is a grossly misleading tool in the hands of plaintiff’s bar. About the time ASECTT v. FMCSA goes to oral argument before the Court of Appeals, the agency proposes to release a revised website for industry comment and subsequent activation. The agency will evidently allow for industry comments, while simultaneously giving carriers (through use of their PIN numbers) a preview of how their own data will appear in the proposed new website.
Unfortunately, the agency’s presentation does not suggest the revised website will address the numerous unresolved data flaws in the system, or disabuse shippers and brokers of their felt need to use percentile rankings to avoid using carriers which the agency has otherwise determined to be fit, willing and able under objective standards.
To the contrary, the agency appears prepared to try to re-substantiate SMS percentile rankings as a measure of ultimate carrier safety, notwithstanding the thorough refutation of the UMTRI study by the Gallo and Gimpel studies, and despite the Iyoob analysis debunking the agency’s previous line graphs showing alleged crash risk. In fact, the agency is working on a proposal to use bar graphs on its revised website to advance the theory that carriers above the monitoring thresholds (golden triangle carriers) are greater crash risks and hence unfit for use by a non-discriminating shipper public.
One bar graph will attempt to show that carriers above the intervention threshold are more likely to have crashes than the national average, with carriers above the thresholds in Unsafe Driving and Hours of Service at greater risk. A second chart will show that the alleged crash risk increases as a carrier’s number of golden triangles increases from 1 to 5. By using the age-old graphing trick of exaggerating the vertical axis, the agency will make the alleged differences between carriers in various triangle categories look far more dramatic than they really are.
The FMCSA’s June 18 presentation on these graphs made clear that the agency, which has yet to release an objective safety fitness determination and has 27 other backed-up rulemakings, is still committed to devoting substantial resources to the basic objective challenged in ASECTT’s lawsuit. That ongoing objective is to promote SMS percentile rankings as benchmarks for shipper and broker use in carrier selection, not as a heuristic tool for the agency’s own use in allocating enforcement resources.
Like the UMTRI study before it, the agency’s latest analysis is based upon an averaging of averages. As in the line graphs previously refuted by Iyoob, the new bar graphs lump together all carriers in all peer groups regardless of fleet size, to create aggregate numbers which may have absolutely no relationship to individual performance. Remember the Iyoob study showing that the UMTRI trend lines looked more like an ink blot than a correlation between golden triangles and crashes when plotted for individual carriers!
The same methodological flaws which invalidated use of the UMTRI study to brand individual carriers will apply to these latest bar graphs. These flaws will be reemphasized because all carriers above an intervention threshold will be aggregated into regardless of the peer group or whether they are in the 66th or 99th percentile.
On further analysis, the proposed bar graph showing that carriers with multiple scores above the intervention threshold are most crash likely, has an additional and equally fatal flaw. Data included in the analysis is based not only on roadside inspections reflecting actual over-the-road performance, but also on violations found in compliance reviews. This creates a chicken and egg problem since carriers most often subjected to compliance reviews are those with high crash ratios in the first place. Additional golden triangles are typically piled on to these carriers as a result of audits, particularly in the lesser BASICs such as drug and alcohol testing violations which are infrequently detected at roadside.
Simply stated, the charts the agency intends to release will place a viscerally appealing tool in the hands of the plaintiffs’ bar and other safety advocates seeking a short-cut method for terrorizing the credulous and branding carriers. It will stigmatize thousands of carriers with de minimis crash scores as bad actors. It will only further highlight the dichotomy between an agency which must make objective findings of fact and a shipping public which it encourages to go into vigilante mode to protect themselves against the perils of vicarious liability.
No tough questions were asked of the agency about continued publication of the scores although the Parker Committee had previously reported that the branding effect of published scores was one of its largest concerns.
We can only hope that the agency will reconsider its evident plans to make matters worse by continuing to emphasize a relativistic safety model as fit for use by the public or the agency. Clearly, ASECTT must remain vigilant and must continue its fight for the objective safety standards mandated by law. The next MCSAC meeting is in the second week of September.
Respectfully submitted,
Mark Andrews Henry Seaton
[…]
