When 2+2=4 is Wrong, the Case Against Irrelevant Travel Forecasting Research

“What?!”, you say.  “2+2=4 is an established mathematical fact.  It cannot be wrong.”  Yes, 2+2=4 has lots of valid applications, such as counting toy blocks.  It can even work for adding up vehicle trips on links.  It does not work for standard deviations of independent random variables, where 2+2=2.828 (approximately).  I am taking issue with researchers who cannot solve the problem needing solving, so they disregard inconvenient aspects of the problem and create a lot of perfectly fine mathematics that can cause incorrect forecasts.

Rest assured, I have seen this issue many times in research by many different people.  This is a broad cultural concern.

First a disclaimer.  I am not a mathematician.  Some people think I am good at math, but I am really just a person who is often adept at applying other people’s math to everyday planning problems.  I am a tinkerer at heart.  If I can get math to do what I want, I don’t really care whether it has been proved conclusively.

What does it matter if irrelevant research gets published?  First, it creates clutter.  Second, it validates incorrect thinking.  Third, it devalues application-oriented research.  Fourth and most importantly, it can lead practitioners astray.

I not am going to accuse any person in particular (maybe you know who you are already), so my first example may be a bit abbreviated and hypothetical.  Here, behavioral realism is paramount.

First Example.  Let’s take the case of travel time unreliability of paths within user equilibrium (UE) traffic assignments. There is a fairly substantial body of literature saying that the value of unreliability is related to the standard deviation of travel time.  Now, the vast majority of this literature is composed of stated preference studies, which are always suspect, and we really don’t know for sure what the functional form should look like in all cases.  Nonetheless, the standard deviation assumption is about as good as we are going to be able to do for a while.

Over the last few years I have seen multiple papers by multiple research teams proposing alternative functional forms, simply because standard deviations are not additive.  Continuing the example, a researcher might choose to say that the value of unreliability is a constant fraction of link travel times.  (I am not aware of anyone doing this specifically, but I would not be surprised if somebody had actually done it.)  An assumption like this has two powerful effects.  First, it enables the researcher to prove convergence of the UE assignment with standard network theory.  Second, it destroys any practical validity of the algorithm.  A practitioner would get erroneous answers and a future researcher would be thrown off the track toward a behaviorally-correct method.

New algorithms that violate well-established behavioral principles are hurtful.

Another Example.  Perhaps you recall the decades-old debate over the error term in the utility equation within discrete choice theory.*  The original derivation of the logit equation postulated that the error term was Gumbel distributed.  The Gumbel distribution seemed like an odd pick to many, so other distributions were considered.  The probit equation, which was based on the Gaussian distribution and had its advocates, was definitely more plausible but far more difficult to estimate.  Importantly, logit came into practice ahead of probit.  I cannot recall whether anybody at the time had convincing evidence as to whether probit was actually better than logit, behaviorally.  We can pretty much agree that probit was more consistent with probability theory, but maybe not so dominatingly as to completely obsolete logit.  We still see papers being published with logit equations at their core.

For my part there are qualitative differences between these two examples.  The logit/probit debate was about statistical principles and the unreliability issue is about human behavior.  I am willing to tolerate mathematical approximations that allow me to do my job, but I am not willing to tolerate mathematics that undermine our understanding of travel and traffic behavior.  If travel forecasting is ever to become a true science we will need to protect what we know, build upon it and replace it when we can do better.

As always, I would like to hear your opinion.

Alan Horowitz, Whitefish Bay, January 10, 2017

*See Boyce and Williams, Forecasting Urban Travel, for a technical history of this debate.