First, I know it’s a hobbyhorse, but instant runoff won’t do anything. In Australia, it hasn’t prevented a two-party system, and in many IRV election scenarios, you can make a candidate win by voting against him. What you want to get a multiparty system is proportional representation…
More to the point, most people have nothing to gain from voting, which means they have nothing to gain from being politically informed. You have to rely on the public’s sense of civic values before you start applying rational choice theory to voting.
A person who doesn’t vote might be socially ostracized, so he needs some mechanism for deciding who to vote for. Sometimes he’ll do it by voting what’s acceptable among his peer group, for example, for the social democratic party if he’s working class; as a corollary, when the peer group has no clear party to vote for, its voter turnout will be lower, and indeed the presence of a working class party tends to increase voter turnout. Other times, he’ll vote based on his perceived interests, but those will again be mediated by the peer group.
People are surprisingly rational. They are informed about what they do, and about any number of hobbies they might acquire to have a life outside work. However, groups are irrational, and voting is more a group exercise than an individual one, even with the secret ballot. A white racist may care very deeply about having health insurance, but if he believes that the correct vote for a member of his (white) peer group is for the (racist, anti-public health) conservative over the (racially inclusive, pro-public health) liberal, he’ll vote conservative. He’s not voting against his interests; his single vote, and even the votes of everyone he knows, will never be enough to tip the election.
That’s why normal political marketing is so weak. It can tip close elections, but not do much more than that. The greatest invention of modern American political marketing, direct mail, is not about rhetorical strategies or language, but about being able to say things to your core supporters that you’d face a massive backlash if you said in public. Political junkies, who are committed to one side and who have a financial, social, or psychological stake in being politically informed, respond to different campaigns from the general population.
Now, Hitler’s a pretty good example here. He was successful at a time when the economy was in shambles and regime support was low; before the Great Depression, his party was a fringe movement. The German who wanted to protest government inaction had two options: the Nazis and the communists. The communists made some gains among socialist voters, but the socialists and communists were never successful outside their working class peer group. The parties that were acceptable among the Protestant middle class and the unemployed were the two liberal parties, the conservative nationalists, and the Nazis, and of those only the Nazis promised fundamental change.
I’m probably going to come off as more of a functionalist than I really am… obviously, things like charisma and political talent matter. That’s why Hitler got so far while the conservative nationalists, who were against the democratic system just as much, became his junior coalition partners. But conversely, American racists managed to hold power for many decades despite producing only one strong leader, George Wallace, who was most active during their final hour.