FMCSA A & I Online
U.S. Dept. of Transportation, Motor Carrier Safety Advisory Council, Federal Motor Carrier Safety Administration U.S. Department of TransportationU.S. Department of TransportationFederal Motor Carrier Safety AdminstrationAnalysis & Information Online
 layout image
 
  Popup arrow Link will open a new browser window

Methodology for State Safety Data Quality

The Methodology for State Safety Data Quality (SSDQ) was developed by FMCSA to evaluate the completeness, timeliness, accuracy, and consistency of the State-reported commercial motor vehicle crash and inspection records in the Motor Carrier Management Information System (MCMIS). The SSDQ evaluation uses a 12-month time period that ends three months prior to the MCMIS snapshot for each measure, unless otherwise stated in the rating description.

Crash and inspection records were used in this evaluation if the date of the event occurred within the 12-month time period, not when the records were uploaded to MCMIS. The quality of this data is evaluated with each monthly snapshot and the States receive ratings of "Good," "Fair," or "Poor" for nine SSDQ Measures. Based on these individual ratings, plus the Overriding Indicator, each State receives an Overall State Rating. The methodology used to determine these ratings is provided below.

Overall State Rating

Considers all nine SSDQ measures and the Overriding Indicator, except measures with a rating of "Insufficient Data." States receive an overall score based on ratings in each of the measures and the Overriding Indicator. A State that has received a "red flag" will be automatically rated "Poor". A State with at least one "Good" crash measure, one "Good" inspection measure, and no "Poor" measures receives a "Good" rating. A State with only one "Poor" measure will receive a "Fair" rating, and any State with two or more "Poor" measures will receive a "Poor" rating. (See image below.)

Overall State Rating is based on ratings in each of the measures and the Overriding Indicator.

The Overall State Rating is determined as follows:

Rating Criteria
Good  Good Rating Minimum of 1 Good Crash Measure, 1 Good Inspection Measure, AND 0 Poor 
Fair  Fair Rating Maximum of 1 Poor 
Poor  Poor Rating 2+ Poor OR Red Flagged 
* States that are red flagged are automatically rated POOR overall.

 

Crash Rating

Considers the five SSDQ crash measures and the Overriding Indicator, except measures with a rating of "Insufficient Data." States receive an overall score based on ratings in each of the crash measures and the Overriding Indicator. A State that has received a "red flag" will be automatically rated "Poor". A State with at least one "Good" measure and no "Poor" measures receives a "Good" rating. A State with only one "Poor" measure will receive a "Fair" rating, and any State with two or more "Poor" measures will receive a "Poor" rating. (See image below.)

The Crash Rating is based on ratings in each of the five SSDQ crash measures and the Overriding Indicator.

The Crash Rating is determined as follows:

Rating Criteria
Good  Good Rating Minimum of 1 Good AND 0 Poor 
Fair  Fair Rating Maximum of 1 Poor 
Poor  Poor Rating 2+ Poor OR Red Flagged 
* States that are red flagged are automatically rated POOR overall.

Crash Record Completeness: Average of Driver and Vehicle Identification Completeness Evaluations

The Crash Record Completeness Measure evaluates fatal and non-fatal crash records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This measure determines a rating based on the completeness of driver and vehicle crash data reported to FMCSA. A State's rating is determined by evaluating the completeness of the driver data and vehicle data separately and then averaging these results together. The completeness of the driver data is determined by the Driver Identification Completeness Evaluation and the completeness of the vehicle data is determined by the Vehicle Identification Completeness Evaluation.

The Crash Record Completeness measure is the average of the Driver and Vehicle Identification Completeness Evaluations
Driver Identification Completeness EvaluationThis evaluation determines the percentage of State-reported fatal and non-fatal crash records in the MCMIS database with complete driver information (i.e., the number of crash records with complete driver information divided by the number of crash records reported) over a 12-month time period. A State-reported crash record is considered complete when the following information is provided: driver license number, driver date-of-birth, driver first name, driver last name, and license class. If any of this information is missing, a record is considered incomplete.
Vehicle Identification Completeness Evaluation This evaluation determines the percentage of State-reported fatal and non-fatal crash records in the MCMIS database with complete vehicle information (i.e., the number of crash records with complete vehicle information divided by the number of crash records reported) over a 12-month time period. A State-reported crash record is considered complete when the following information is provided: vehicle identification number, license plate number, vehicle configuration, cargo body type, and gross vehicle weight rating. If any of this information is missing, a record is considered incomplete.

The Crash Record Completeness rating is determined as follows:

Rating Criteria
Good  Good Rating Percentage of completed driver and vehicle information is >= 85%
Fair  Fair Rating Percentage of completed driver and vehicle information is 70 - 84%
Poor  Poor Rating Percentage of completed driver and vehicle information is < 70%

 

Non-Fatal Crash Completeness Measure

All States are responsible for reporting fatal and non-fatal crash records to FMCSA that meet the standard reporting criteria. FMCSA uses these records to support its mission to reduce crashes, injuries, and fatalities involving large trucks and buses on our nation's roadways. Because of the complexity of State crash reporting systems, it is often difficult for States to determine if all reportable non-fatal crash records have been submitted to FMCSA. State-specific evaluations of crash reporting have determined significant underreporting and prompted the need to develop non-fatal crash record benchmarks for each State. The NFCC measure is intended to serve as a guideline to assess whether a State's non-fatal crash reporting falls within an expected range.

 View the Interactive Guide to learn about the NFCC evaluation process

The NFCC measure determines a State rating by comparing the number of State-reported non-fatal crash records in MCMIS to an expected range of non-fatal crash records generated by a statistical model. The input data used by the model, how the data are processed through the model, and the evaluation and final determination of a State's NFCC rating are described in a 4-step approach.

Step 1: Input State Fatal Data

The measure's algorithm utilizes a relationship between the numbers of fatal crash involvements to the number of non-fatal crash record involvements. State's fatal crash records reported to MCMIS are used as an input value to this measure. This record set represents interstate and intrastate motor carriers and includes large truck and bus vehicle types. The MCMIS fatal crash records cover a 12-month time period that ends six months prior to the MCMIS snapshot date.

Step 2: Process Data Through a Statistical Model

It is hypothesized that a ratio of fatal crash involvements to non-fatal reportable involvements exists that is independent of any State’s data system and applies across all the States. Since the number of fatal crash involvements is generally well known, it is then possible to predict the number of non-fatal crash involvements that occur in a State. The NFCC measure utilizes a simple log-linear regression model that translates the number of fatal crash involvements in MCMIS to an expected range of non-fatal crash record involvements. The State fatal data is input to the statistical model to generate estimates of non-fatal crash involvements.

Two sets of prediction intervals are generated by the model to provide ranges of non-fatal crash involvements. The prediction intervals are expressed in terms of the probability that the number of reportable non-fatal crash involvements fall within the prediction interval. The model was programmed to output 99% and 90% prediction intervals. The prediction intervals are used to define expected data ranges, and each range has an upper and lower boundary.

The model incorporates an urbanization factor to account for the proportion of rural to urban commercial vehicle travel in a State. The purpose of the adjustment is to account for any variation in the ratio of fatal to non-fatal reportable crash involvements that may be due to variations in the proportion of rural or urban commercial motor vehicle travel in a State.

The document “A New Model of Crash Severities Reportable to the MCMIS Crash File” explains more about the statistical model used by this measure.

Step 3: Compare State Non-Fatal Data to Model Results

The State-reported number of non-fatal crash records in MCMIS are compared against the expected data ranges output by the model. It is determined where the MCMIS non-fatal crash records fall within the expected data ranges. This determination is used to assign each State a base rating of 'good', 'fair', 'poor', or 'insufficient data'. The base rating assignments are one part of the final rating assignment process.

The diagram below illustrates the rating ranges based on the expected ranges generated from the statistical model. States with few fatal crash records may yield less predictable output and may be assigned an 'insufficient data' rating.

Rating Ranges
Under Reporting   Over Reporting
Poor * Fair Good Fair
<
Lower Boundary
of 99% PI
>=
Lower Boundary
of 99% PI
AND
<
Lower Boundary
of 90% PI
>=
Lower Boundary
of 90% PI
AND
<=
Upper Boundary
of 99% PI
>
Upper Boundary
of 99% PI

* Assign a base rating of 'Insufficient Data' if Reported MCMIS Fatal Records is less than 15
   AND Non-Fatal Records is within the Poor Rating Range

The base rating assignments are defined as follows.

 

Base Rating Assignments: 

Good rating range: Reported Non-Fatal crash records (from MCMIS) is greater than or equal to lower boundary of 90% prediction interval AND less than or equal to upper boundary of 99% prediction interval

Fair rating range Reported Non-Fatal crash records (from MCMIS) is greater than or equal to lower boundary of 99% prediction interval AND less than lower boundary of 90% prediction interval (under reporting) OR
Reported Non-Fatal crash records is greater than upper boundary of 99% prediction interval (over reporting)

Poor rating range: Reported Non-Fatal crash records (from MCMIS) is less than lower boundary of 99% prediction interval AND MCMIS fatal crash records is greater than or equal to 15 (under reporting)

Insufficient Data rating range: Reported Non-Fatal crash records (from MCMIS) is less than lower boundary of 99% prediction interval AND MCMIS Fatal crash records is < 15

Step 4: Determine Rating Results

The NFCC rating is determined by reviewing the base rating results and the most recent fatal crash completeness (FCC) measure rating for each State. Both of these ratings are important for the assignment of the final NFCC rating.

The FCC measure rating is considered since the model's output is using State-reported fatal crash records to generate estimated non-fatal crash record values. If a State's FCC rating is 'good' or 'fair', it is assumed their MCMIS fatal crash reporting is sufficient to estimate the expected number of non-fatal crash records. In those cases, the NFCC base rating is regarded as the final NFCC measure rating.

If a State's FCC rating is either 'poor' or 'insufficient data', it is assumed their MCMIS fatal crash reporting is insufficient due to inadequate reporting or too few fatal crash involvements occurring in the State. An “override” is administered, meaning, the base rating cannot be used. The override will result in an NFCC rating of either 'poor' or 'insufficient data' due to incomplete MCMIS fatal reporting.

The final Non-Fatal Crash Completeness rating is determined as follows:

Rating Criteria
Good  Good Rating Reported MCMIS non-fatal records fall within Good rating range AND FCC rating is Good or Fair
Fair  Fair Rating Reported MCMIS non-fatal records fall within Fair rating range AND FCC rating is Good or Fair
Poor  Poor Rating Reported MCMIS non-fatal records fall within Poor rating range AND FCC rating is Good, Fair or Poor
Insufficient Data Insufficient Data
Base rating is assigned Insufficient Data OR
Reported MCMIS non-fatal records fall within Good or Fair rating range AND FCC rating is Poor OR
FCC rating is Insufficient Data

Fatal Crash Completeness: Percentage of Fatal Crash Records Reported

The Fatal Crash Completeness Measure evaluates only those records that represent large trucks involved in fatal crashes that occurred within the calendar year. This measure determines a rating based on a comparison of the number of State-reported fatal crash records in MCMIS to the number of fatal crash records reported in the Fatality Analysis Reporting System (FARS). FARS is the national database of fatal motor vehicle crashes maintained by the National Highway Traffic Safety Administration (NHTSA).

The Fatal Crash Completeness rating is determined as follows:

Rating Criteria
Good  Good Rating MCMIS as a % of FARS is >= 90%
Fair  Fair Rating MCMIS as a % of FARS is 80 - 89%
Poor  Poor Rating MCMIS as a % of FARS is < 80%
Insufficient Data  Insufficient Data State has < 15 FARS records AND MCMIS as a % of FARS is < 80%

Crash Timeliness: Percentage of Crash Records Reported within 90 Days

The Crash Timeliness Measure evaluates fatal and non-fatal crash records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This measure determines a rating based on the percentage of crash records reported to FMCSA within 90 days over a 12-month period.

The Crash Timeliness rating is determined as follows:

Rating Criteria
Good  Good Rating Percentage reported within 90 Days is >= 90%
Fair  Fair Rating Percentage reported within 90 Days is 65 - 89%
Poor  Poor Rating Percentage reported within 90 Days is < 65%
Insufficient Data  Insufficient Data State has < 15 records reported in current timeframe AND percentage reported within 90 Days is < 65%

 

Crash Accuracy: Percentage of Matched Crash Records

The Crash Accuracy Measure evaluates fatal and non-fatal crash records that represent interstate carriers and intrastate carriers transporting hazardous material and includes large truck and bus vehicle types. This measure determines a rating based on the percentage of crash records reported by the State over a 12-month period that were matched to a company registered in MCMIS. (Crash records entered per FMCSA's "Procedures for Entering Crashes without Carrier Identification into SAFETYNET" are not evaluated by this measure.)

The Crash Accuracy rating is determined as follows:

Rating Criteria
Good  Good Rating Percentage of matched records is >= 95%
Fair  Fair Rating Percentage of matched records is 85 - 94%
Poor  Poor Rating Percentage of matched records is < 85%
Insufficient Data  Insufficient Data State has < 15 records reported in current timeframe AND percentage of matched records is < 85%

Crash Consistency †: Percentage of State-Reported Non-Fatal Crash Records

The Crash Consistency Overriding Indicator evaluates non-fatal crash records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This "red flag" indicates States that have reported less than 50% of non-fatal crash records for the current 12-month period compared to the yearly average, based on the previous 36-months.

The Crash Consistency Overriding Indicator "flag" is determined as follows:

Rating Criteria
No Flag    Estimate Reported is >= 50%
Red Flag  Red Flag Estimate Reported is < 50%
Insufficient Data  Insufficient Data State has < 15 records reported in current timeframe AND
State has < 15 records reported in previous 3 year average AND
Estimate Reported is <= 50%

† States that have an obvious and significant decline in crash record reporting will be categorized as Poor in the Overall State Rating and Crash Rating, without regard to their rating on other measures.

Inspection Record Completeness: Average of Driver and Vehicle Identification Completeness Evaluations

The Inspection Record Completeness Measure evaluates level 1, 2, and 3 roadside inspection records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This measure determines a rating based on the completeness of driver and vehicle inspection data reported to FMCSA. A State's rating is determined by evaluating the completeness of the driver data and vehicle data separately and then averaging these results together. The completeness of four (4) driver data elements are determined by the Driver Identification Completeness Evaluation and the completeness of two (2) vehicle data elements are determined by the Vehicle Identification Completeness Evaluation.

The Crash Record Completeness measure is the average of the Driver and Vehicle Identification Completeness Evaluations.
Driver Identification Completeness Evaluation This evaluation determines the percentage of State-reported level 1, 2, and 3 roadside inspection records in the MCMIS database with complete driver information (i.e., the number of inspection records with complete driver information divided by the number of inspection records reported) over a 12-month time period. A State-reported inspection record is considered complete when the following information is provided: driver license number, driver date-of-birth, driver first name, and driver last name. If any of this information is missing, a record is considered incomplete.
Vehicle Identification Completeness Evaluation This evaluation determines the percentage of State-reported level 1, 2, and 3 roadside inspection records in the MCMIS database with complete vehicle information (i.e., the number of inspection records with complete vehicle information divided by the number of inspection records reported) over a 12-month time period. A State-reported inspection record is considered complete when the following information is provided: license plate number and gross vehicle weight rating. If any of this information is missing, a record is considered incomplete.

The Inspection Record Completeness rating is determined as follows:

Rating Criteria
Good  Good Rating Percentage of completed driver and vehicle information is >= 85%
Fair  Fair Rating Percentage of completed driver and vehicle information is 70 - 84%
Poor  Poor Rating Percentage of completed driver and vehicle information is < 70%

 

Inspection VIN Accuracy: Percentage of Valid Vehicle Identification Numbers Reported on the First Vehicle Unit within Inspection Records

The Inspection VIN Accuracy Measure evaluates level 1-6 roadside inspection records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This measure determines a rating based on the completeness and accuracy of the vehicle identification number reported on the first vehicle unit reported to FMCSA – all trailing units are excluded from this measure. A State’s rating is determined by evaluating the 17-character VIN using the "checksum" digit in the 9th character position. The checksum digit is used to determine if the VIN is accurate based upon an algorithm that uses the other 16 characters in the VIN. Any VIN with invalid characters (i.e. I, O, or Q) or an incomplete field (i.e. less than 17 characters) does not pass the checksum algorithm and is invalid. For this measure, records with all the same numbers (i.e. 99999999999999999) are also counted as invalid.

The Inspection VIN Accuracy rating is determined as follows:

Rating Criteria
Good  Good Rating Percentage of completed and accurate VIN is >= 85%
Fair  Fair Rating Percentage of completed and accurate VIN is 70 - 84%
Poor  Poor Rating Percentage of completed and accurate VIN is < 70%

Inspection Timeliness: Percentage of Inspection Records Reported within 21 Days

The Inspection Timeliness Measure evaluates inspection records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This measure determines a rating based on the percentage of inspection records reported to FMCSA within 21 days over a 12-month period.

The Inspection Timeliness rating is determined as follows:

Rating Criteria
Good  Good Rating Percentage reported within 21 Days is >= 90%
Fair  Fair Rating Percentage reported within 21 Days is 65 - 89%
Poor  Poor Rating Percentage reported within 21 Days is < 65%

Inspection Accuracy: Percentage of Matched Inspection Records

The Inspection Accuracy Measure evaluates inspection records that represent interstate carriers and intrastate carriers transporting hazardous material and includes large truck and bus vehicle types. This measure determines a rating based on the percentage of inspection records reported by the States over a 12-month period that were matched to a company registered in MCMIS.

The Inspection Accuracy rating is determined as follows:

Rating Criteria
Good  Good Rating Percentage of matched records is >= 95%
Fair  Fair Rating Percentage of matched records is 85 - 94%
Poor  Poor Rating Percentage of matched records is < 85%

 

  • Improvements or changes made to the "State Safety Data Quality" methodology are documented below.

    • January 27, 2012
      • The Non-Fatal Crash Completeness measure’s evaluation process was modified by adding additional States and more recent State crash data to the model, adjusting the model based on the amount of rural to urban traffic in each State, and modifying the calculation for determining the measure's rating.
         View the Interactive Guide to learn about the NFCC evaluation process
    • September 24, 2010
      • Two (2) new measures were added to the methodology, which focus on inspection reporting. These measures evaluate the completeness of driver and vehicle inspection data (Inspection Record Completeness Measure), as well as the completeness and accuracy of inspection record vehicle identification numbers (VINs) (Inspection VIN Accuracy Measure).
      • The Crash and Inspection Timeliness measures’ evaluation criteria were modified. The "good", "fair" and "poor" evaluation criteria have each increased by a value of five (5).
      • The Overall State Rating was modified to be based upon five (5) crash measures, four (4) inspection measures and one crash indicator.
      • The Overall State Rating’s evaluation criteria were modified. A state will receive a "good" overall rating if it has at least one "good" crash measure, one "good" inspection measure, and no "poor" measures. Those states that do not meet the "good" measure criteria and do not have any "poor" measures will receive a "fair" rating. The "poor" criteria have not changed.
    • October 30, 2007
      • Two (2) new measures were added to the methodology. These measures evaluate the completeness of driver and vehicle crash data (Crash Record Completeness measure) and the complete reporting of non-fatal crash records (Non-Fatal Crash Completeness measure).
      • A Crash Rating was added to evaluate only crash data quality.
    • June 23, 2006
      • The Crash Accuracy measure’s evaluation process was modified. The crash accuracy analysis now excludes all "Carrier Non-Identifiable" crash records. This modification prevents records that are entered into the Motor Carrier Management Information System using FMCSA's guidelines without carrier identification from being included in the calculation of the measure.
    • March 31, 2006
      • The Crash and Inspection Timeliness measures’ evaluation processes were modified. The timeliness analysis now includes both 'add' and 'change' records. This modification allows all crash and inspection records uploaded to MCMIS within a specific timeframe to be evaluated. The previous methodology only included 'add' records.
    • December 23, 2004
      • The Crash and Inspection Accuracy measures’ evaluation processes were modified. The analysis now includes interstate carriers and intrastate hazardous material carrier records to determine the percentage of records matched to a motor carrier in MCMIS. The previous methodology only included interstate carriers.
    •  A motor vehicle with seats for at least nine (9) people, including the driver's seat.
    •  The 9th digit of a 17-character VIN can be used to determine if the field is valid.
    •  A crash where one or more persons die within 30 days of the crash. The fatality does not have to occur at the scene of the crash. It includes any person involved in the crash, including pedestrians and bicyclists, as well as occupants of the passenger cars and trucks.
    •  Carriers that transport a commodity outside the State of its place of business.
    •  Carriers that transport a commodity only within the State of its place of business.
    •  Any truck having a gross vehicle weight rating (GVWR) of more than 10,000 pounds or a gross combination weight rating (GCWR) over 10,000 pounds.
    •  A statistical tool where existing data are used to define a relationship between two sets of numbers, so that in the future, one set can be reliably predicted from the other. In reference to the NFCC measure, verified fatal and non-fatal crash record data were used to define a relationship for predicting an expected range of non-fatal crash records from a known quantity of fatal crash records.
    •  The calculation of a State’s NFCC measure rating is determined by reviewing its Base Rating to its fatal crash completeness measure rating. The Base Rating establishes a good, fair, poor or insufficient data rating range for each State. Each designation is determined by where a State’s non-fatal crash reporting falls.
    •  A crash where one or more persons has non-fatal injuries requiring transportation by a vehicle for the purpose of obtaining immediate medical attention; or one or more of the vehicles were towed away from the scene due to "disabling damage". The towed vehicle need not be the commercial motor vehicle involved in the crash.
    •  An estimated range of values, as defined by a lower and upper value, is determined by a mathematical model and the probability that another number will fall between them. In reference to the NFCC measure, the log-linear regression model generates a unique interval of non-fatal crash records.
    •  A ratio of rural to urban commercial motor vehicle (CMV) travel in the State as calculated from the Federal Highway Administration’s data in the Annual Highway Statistics publication. For each State, the RU factor is calculated by averaging the most recent three years of annual rural CMV travel miles by the average of the most recent three years of annual urban CMV travel miles.
Department of Transportation logo

Feedback | Privacy Policy | USA.gov | Freedom of Information Act (FOIA) | Accessibility | OIG Hotline | Web Policies and Important Links | Site Map | Plug-ins

Federal Motor Carrier Safety Administration
1200 New Jersey Avenue SE, Washington, DC 20590 • 1-800-832-5660 • TTY: 1-800-877-8339 • Field Office Contacts