Polytomous (Multinomial) and Ordinal Logistic Models

If your dependent variable is continuous or near continuous you use a regression technique; if the dependent variable is binary you can use a logistic regression. Often times; however, what we are trying to model is neither continuous nor binary. Multi-level dependent variables are common occurrences in the real world. Examples of Multi-level dependent variables are:

1: ‘yes’, ‘no’ or ‘does not respond’

2: ‘high’, ‘medium’ or ‘low’ levels of agreement.

These can sometimes be modeled using binary models. You can collapse two categories together, say no and does not respond resulting in a binary choice model. This however can obscure relationships especially is the groups are incorrectly formed or there are several potential outcomes that do not group together logically (i.e. does medium go with low or high?).

There are two main types of models for handling mutually-exclusive, multi-level dependent variables, one for ordinal outcomes and the other is for nominal outcomes. Nominal outcomes are where the ordering of the outcomes does not matter like in example 1. Ordinal outcomes are when the order matters like in example 2. For nominal outcomes you can use Polytomous (Multinomial) models and for ordinal outcomes you can use Ordinal or Polytomous models.

Polytomous models are similar to standard logistic models but instead of one log odds ratio to estimate you have multiple, one for each response. Ordinal Models are stricter in there assumption than Polytomous models namely it assumes the response variables have a natural ordering. Like the Polytomous models you estimate a log odds for each response.

Both of the models have the same assumption of the standard logistic model plus a few more. One new assumption for both models is that the outcomes are mutually exclusive. In the two examples above it is clear the categories are mutually exclusive. But, consider this example: who do you like: Candidate A, B, C? Someone could easily like more than one candidate. In these cases when this assumption is too restrictive you may use data mining approaches such as SVM or ANN.

Another assumption is independence of irrelevant alternatives (IIA). Since the log odds for each response is pitted against one another it is assumed people are behaving rationally. An example of a violation of IIA is if someone prefers candidate A to B, B the C and C to A. Given the preferences A to B and B to C a rational person would prefer A to C. If IIA is violated it can make interpreting the log odds impossible. For a detailed discussion go to:

http://en.wikipedia.org/wiki/Independence_of_irrelevant_alternatives.
www.stat.psu.edu/~jglenn/stat504/08_multilog/10_multilog_logits.htm

www2.chass.ncsu.edu/garson/PA765/logistic.htm

www.mrc-bsu.cam.ac.uk/bugs/documentation/exampVol2/node21.html

en.wikipedia.org/wiki/Logistic_regression

nlp.stanford.edu/IR-book/html/htmledition/node189.html

www.ats.ucla.edu/STAT/mult_pkg/perspective/v18n2p25.htm