A Desire for Increased Knowledge of Errors in Ground Data for Project-Level Forecasting and Model Validation

There is a smattering of published information on errors in ground data, but this topic needs much more attention by travel modelers.

I am narrowly restricting my definition of an “error” to a difference between an actual and a measured ground data item.  I refer to a problem which causes a model to fail to match reality as a “mistake”.

I will start off this discussion by concentrating on highway data; I do not want us to ignore transit or other passenger modes or freight.  I do not want us to ignore errors in surveys, either.

My very first exposure to ignorance of errors in ground data occurred in the early 1990’s when a traffic engineer for the State of South Dakota complained that the Sioux Falls model could not validate any closer than 10% of highway ground counts.  He wanted 5%.  Ground count data is not usually good enough to meet a 10% validation threshold, much less a 5% threshold.  To this engineer, ground counts were perfect.  Witness the graph below taken from the Travel Model Validation and Reasonableness Checking Manual, 2nd Edition, page 9-5.

The solid line in this graph is recommended (by me and maybe a few others, as well) in a number of documents as a standard for the minimum plausible error that should be observed in any travel model validation.  Interestingly, the FDOT traffic counting program could not even achieve these results.  FDOT achieved just a 12% to 15% coefficient of variation, regardless of the volume range, as seen with the pink dots.  NCHRP Report 765 also recommends the solid line as a warning for when synthetic OD table estimation is overfitting traffic counts.  A better fit than this solid line runs the risk of forcing our travel forecasts to mimic errors in traffic counts.

I have not seen a similar graph for turning movements or origin-destination flows, but almost everybody who thinks about such things assumes these errors are much worse than daily road mainline counts.  In addition, I have not seen much on how errors vary across hours in a day or across vehicle classes.

Given the criticalness of ground data to our forecasting process, shouldn’t we know much more about the errors in these data?

Some errors arise through equipment or human inaccuracies, but a large portion of errors are consequences of unpredictable events.  We can partially adjust for day-of-week, season and functional class, but we cannot adjust most ground counts for how a given road varies in character from others in its cluster or for traffic disruptions on or near the count station.  A 48-hour count would most likely vary considerably from an ADT taken from a continuous traffic counter, had such a counter been in place.

So instead of only trying to fix errors in traffic counts at their source, it might be a better strategy to figure ways to minimize the impacts of those errors.  We already do this to some extent when certain agencies validate their models to screenline counts rather than to individual road counts.

As a start, maybe you can help me by pointing out documentation of errors inherent in different types of counting situations and equipment.  If there is enough prior work, perhaps an NCHRP Synthesis study could pull all of this information together for us.  However, if existing information is incomplete or sparse, a full NCHRP Report makes more sense.

I have many burning questions.  How do volume counting errors propagate through a synthetic OD table estimation process?  How much error is there in an OD table count when carried out by various vehicle re-identification technologies?  Ditto for cell phone ping records.  How do counting errors vary by time of day?  Is there a relationship between vehicle mix and counting errors?  What is the effect of refinement on error-prone turning movement counts, given that mainline counts also have their own errors?  How much error do we expect to see in spot speeds and travel times through corridors?  Are spot speed data good enough to assist OD table estimation in some reasonable fashion?  How accurate are truck counts?  Which probability distributions best describe these errors?

NCHRP Reports 255 and 765 suggested that ground counts were better than forecasts, but few of the methods in those reports explicitly considered error in those ground counts.  Are there additional tricks to further mitigate those errors?

As seen in my earlier blog article comparing pivot point to the gravity model for OD table extrapolation, errors can affect techniques in different ways.  The ability of a technique to withstand sizeable errors should probably be a selection criterion.

As always, I am interested in hearing your comments.

Alan Horowitz, Whitefish Bay, May 3, 2017